Comparing two assays for clinical genomic profiling: the devil is in the data

We read with concern the paper "Evaluation and comparison of two commercially available targeted next-generation sequencing platforms to assist oncology decisionmaking". 1 The study directly compared results for the Paradigm Cancer Diagnostic test to the FoundationOne test for formalin-fixed, paraffin-embedded specimen pairs from 21 advanced cancer cases.We believe this study is fundamentally flawed, misleading, and potentially dangerous for patient care, for the reasons outlined herein.The paper neglected to address innumerable discordances between a rigorously analytically validated test (FoundationOne) and the experimental assay.It erroneously ascribes categorization of many genomic alterations detected on FoundationOne as "none", when in fact available drugs have demonstrated activity or mechanism-based clinical trials exist, and it claims high levels of actionability based on the results of RNA-expression profiling of a single gene -TOPO2A.This study is notable for a remarkable lack of concordance between the genomic alterations detected on each platform, even in genes common to both assays.For the majority of specimens, there were no genomic alterations in common, and only six of 21 samples shared a single alteration, matching at the gene level.Paradigm DNAbased testing found only two cases wherein a result was labeled by them as "commercially available" or "clinical trial", an EGFR mutation in a lung adenocarcinoma, and a KIT mutation in a colon adenocarcinoma.This KIT mutation was not detected by FoundationOne.Similarly, lack of concordance is noted for mutations in ARID1A (case 1), PDGFRA and PI3K (case 2), KRAS, PI3K and PTEN (case 5), PTEN and ARID1A (case 6), EGFR amplification (case 9), ERBB2 (cases 12, 14), and PI3K (case 19).Given the significant discordance in the genomic alterations detected and reported, it must reasonably be concluded that at least one assay has extremely poor mutation-detection performance.FoundationOne underwent an extensive 2-year analytic validation study.This study rigorously demonstrated the test's high-performance characteristics. 2 The Paradigm assay has no published analytic validation studies.The lack of concordance between the two platforms calls into serious question the performance of the Paradigm assay and the validity of the data.Clinical activity and/or mechanism-directed trials exist for genomic (DNA-based) alterations in PTEN, 3 RICTOR, 4 CDK6 (NCT02187783), FGFR4 (NCT02325739), ERRFI1, 5 FBXW7, 6 and PI3KR1 (NCT01971515).The "actionable target" determination, as displayed in Table 2, also reveals a failure to acknowledge KRAS as a gene for which therapeutic approaches exist in clinical trials, and for which prior genomically driven trials have shown selected activity.7 Although an acknowledged therapeutic