Clinical Metagenomics—from Proof-of-Concept to Routine Use

This is the authors' abstract. We don't add key points for this paper.

Metagenomics was first used for pathogen discovery in suspected outbreaks and to study microbial communities. Diagnostic applications were initially accessible only to a few specialized laboratories and used in individual, complex patients (1, 2). Reduced cost, availability of sequencers, and more rapid, user-friendly data analysis solutions broadened access (3–5). Subsequent studies in dozens of patients with negative conventional tests demonstrated increased yield (6). More recently, streamlined workflows and costs in the range of multiplex PCR panels have enabled larger studies in more common conditions (7). Studies often focus on analytical performance by comparing metagenomics to a battery of culture, PCR, and antigen tests comprising today’s standard of care. While infections with exotic pathogens receive most attention, a much larger number of patients stand to benefit from earlier diagnosis of infections for which specific tests (and treatment) exist but are not ordered until late, or for pathogens that take days to weeks to grow in culture. Thus, the real potential of clinical metagenomics will only be realized when testing is performed more routinely and earlier during workup, when effective management can impact outcomes. Despite growing evidence and interest, adoption still lags behind other next-generation sequencing (NGS)-based tests, such as rapid genome sequencing for the diagnosis of inherited disorders in newborns, non-invasive prenatal testing, and oncology. In this issue, Tao et al. (8) reviewed results of clinical metagenomics used during routine care in >500 hospitalized children (274 requiring intensive care). The study demonstrated clinical validity and remaining barriers for broader adoption, including the need to better characterize test performance, standardize workflows, and streamline result interpretation. Establishing analytical performance of metagenomics tests is complex and often varies by pathogen, sample type, and specific analytical methods. Metagenomics tests can detect thousands of microorganisms (whether harmful, beneficial, or of no impact to the patient), which makes it difficult to assess performance in the absence of reference methods with similar breadth and resolution. There are >1000 human pathogens with highly diverse biological properties that include: physical properties of genomes (single- or double-stranded, RNA or DNA); genome sizes ranging from a few thousand to tens of millions of base pairs (bp); and genomes protected by simple protein capsids, lipid membranes, or hardy cell walls (9). Genomic similarity is poorly correlated with species definitions, often requiring pathogen-specific approaches. For some pathogens, strains of the same species (leading to the same diagnosis) may share <70% genome identity, while others share >99% of their genome sequence with entirely different pathogens or even non-pathogenic microbes. Thus, biological characteristics and taxonomic realities pose substantial and often idiosyncratic challenges for nucleic acid (NA) extraction, library preparation, and sequence-based pathogen identification, and eventually analytical performance. Even if known, analytical performance is only a proxy for clinical validity. By comparing results against clinical diagnoses, Tao et al. focused instead on clinical validity, which requires interpretation of laboratory, imaging, and clinical information. Clinical validity represents the “ability of a test to correctly classify a patient with respect to a diagnostic, prognostic, or predictive category” and depends on the patient selection criteria and clinical endpoints (10). Children are vulnerable to infections (common and unusual) making them an important population to study. Overall, metagenomics was more clinically sensitive (74%) but less specific (59%) than conventional diagnostics (38% and 83%, respectively). Sensitivity was particularly improved for fungal infections (94% vs 44% by traditional means) addressing an important need. Rapid and sensitive detection of fungal infections is critical for early, targeted treatment in immunocompromised patients and limits the unnecessary use of antifungals. Multiple potential pathogens were detected in 53% of metagenomics-positive samples, highlighting the importance of rational test design and interpretation. Of 303 patients with true-positive results, 197 were clinically diagnosed with an infection by a single pathogen. In 176 (89.3%) of these, the causative pathogen was also most abundant as measured by normalized sequencing read count and in another 17 (8.6%), the pathogen was second most abundant. These results are consistent with the traditional approach in clinical microbiology laboratories to assign greater significance to pathogens that are more abundant in culture. Importantly, metagenomic testing was performed in-house with an average turnaround time of 1.3 days, demonstrating benefits over send-out testing. For results of metagenomics studies to be generalizable and to limit development in the growing number of laboratories performing clinical metagenomics, workflows need to be more standardized. Test performance is heavily influenced by several factors: the specimen’s NA concentration and composition (pathogen, other microbial, and host); collection, transport, and storage; extraction efficiency for different pathogens; sequencing depth and read length; and databases, algorithms, and detection thresholds during data analysis. Extraction efficiency and yield for host and microbial cells differ between methods, which can significantly impact performance. Adequate quality control, including of extraction steps, is required to ensure consistent results. Unfortunately, the reality today is that laboratories use methods that are often unique to their research or diagnostic practice. Even at the same laboratory, varying methods may be used. In the study by Tao et al. DNA sequencing (DNA-seq) was used for 65% of samples, RNA sequencing (RNA-seq) for 6%, and DNA-seq plus RNA-seq for 29%. In the latter group, some pathogens were only detected by RNA-seq, highlighting how method selection impacts results. Standardization is even more critical for data analysis. Broad and accurate pathogen detection hinges on comprehensive, balanced databases with correctly annotated reference sequences, classification algorithms, and optimized detection and reporting thresholds. There is tremendous bias in the number of available genome sequences per pathogen (hundreds of thousands for some, no complete genomes for many others). Mis-annotation of microbial sequences is widespread in public databases requiring extensive curation. There are no accepted standards, and most studies are based on laboratory-specific databases. Detection and reporting thresholds determine sensitivity and specificity and are difficult to optimize. With shotgun metagenomics, pathogens are often reported based on detection of small portions of their genomes. For example, Tao et al. discuss the impact of 2 different thresholds: RPM > 3 and RPM > 10 (RPM, pathogen reads per million total reads). With around 20 million total reads/sample, this represents 60 and 200 pathogen reads, respectively, for a positive result, or 9000 to 30 000 bp of pathogen sequence (for 150 bp reads). While this can represent full genomes for some viruses, it equals <1% of a typical bacterial genome. Since reads are usually randomly scattered across the pathogen genome, they yield widely differing value, dependent on how informative the region is for pathogen identification and characterization. Therefore, thresholds based on normalized read counts alone are complex to derive at best and often flawed, especially as they need to be pathogen specific as discussed in this manuscript. Details of data analysis steps and detection criteria are frequently unreported in manuscripts, making results difficult to interpret and reproduce. Standardization of laboratory and data analysis steps will accelerate evidence generation, increase external validity, and facilitate adoption of clinical metagenomics. Conceptually, microorganism detection by metagenomics is analogous to variant identification in NGS tests for inherited disorders and cancers (“secondary analysis”), which requires further prioritization and interpretation (“tertiary analysis”) (11). While in some instances, detection of a given pathogen is sufficient for management decisions, interpretation of results often requires correlation with other laboratory, imaging, and clinical data. This is particularly true when multiple potential pathogens are detected. The study by Tao et al. detected multiple potential pathogens in 53% of positive samples. A common approach to prioritize and assign significance is based on pathogen abundance (12). Tao et al. use normalized read counts as a surrogate for pathogen abundance and showed correlation with the clinical interpretation. NGS data can also be used for absolute pathogen quantification, which allows for prioritization or even quantitative thresholds. This study included an uninfected control group to establish specificity. Potential pathogens detected in controls were defined as “false positives,” highlighting the need for appropriate result interpretation. For now, laboratories need to be prepared to discuss metagenomics results with clinicians. As has been the case with inherited disorders (13), this tertiary analysis will likely be further automated leveraging ancillary information from the metagenomics results and data from clinical studies. Unlike culture and PCR, metagenomics can provide a wealth of additional information about pathogens (e.g., quantity, antimicrobial resistance, and pathogenicity factors), samples (e.g., composition of microbiota and host cells), and patients (e.g., host response) for interpretation (14–17). Quantitative thresholds based on high-resolution and objective data predicting clinical phenotypes may prove helpful, especially for body sites that contain a resident microbiota. The authors concluded that “On the whole, the most informative samples for diagnostic sequencing are obtained from the primary site of infection, if possible” but more studies are needed to identify clinical scenarios for which more accessible specimens such as plasma may be sufficient (18). As Tao et al. demonstrate, in-house metagenomics testing can shorten turnaround times, a critical feature for any infectious disease test. The next frontier will be to demonstrate clinical utility and cost effectiveness in larger, prospective trials using antibiotic stewardship (e.g., antibiotic usage and time to adequate therapy) and clinical outcomes (e.g., length of stay, ICU admission rate, and mortality). In the meantime, studies like this one provide evidence for adjunct use of clinical metagenomics earlier in the diagnostic workup, combining the specificity of culture with the breadth and sensitivity of metagenomics. All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest R. Schlaberg, co-founder and employee of IDbyDNA Inc. R. Schlaberg, IDbyDNA Inc., Illumina. R. Schlaberg, IDbyDNA Inc. None declared. R. Schlaberg, NIH, University of Utah. None declared. R. Schlaberg, several patents applications related to metagenomics. R. Schlaberg, IDbyDNA Inc., support for attending meetings and/or travel from Association of Molecular Pathologists, Cambridge Healthtech Institute, Global Microbial Identifiers, DGKL (German Society for clinical chemistry and laboratory medicine), and the European Commission JRC.