Development, Validation, and Implementation of an Augmented Multiwell, Multitarget Quantitative PCR for the Analysis of Human Papillomavirus Genotyping through Software Automation, Data Science, and Artificial Intelligence

The value of human papillomavirus (HPV) testing for cervical cancer screening is well established; however, its use as a primary screening option or as a reflex test after atypical cytology results is now gaining wider acceptance. The importance of full genotyping and viral load determination has been demonstrated to enhance the clinical understanding of the viral infection progression during follow-up or after treatment, thereby providing clinicians with supplementary tools for optimized patient management. We developed a new analysis method for the RIATOL quantitative PCR assay, and validated and implemented it in the laboratory of clinical molecular pathology at Algemeen Medisch Laboratorium (AML), under national accreditation and following the International Organization for Standardization guidelines. This study presents the successful validation of a high-throughput, multitarget HPV analysis method, with enhanced accuracy on both qualitative and quantitative end results. This is achieved by software standardization and automation of PCR curve analysis and interpretation, using data science and artificial intelligence. Moreover, the user-centric functionality of the platform was demonstrated to enhance both staff training and routine analysis workflows, thereby saving time and laboratory personnel resources. Overall, the integration of the FastFinder plugin semi-automatic analysis algorithm with the RIATOL real-time quantitative PCR assay proved to be a remarkable advancement in high-throughput HPV quantification, with demonstrated capability to provide highly accurate clinical-grade results and to reduce manual variability and analysis time. The value of human papillomavirus (HPV) testing for cervical cancer screening is well established; however, its use as a primary screening option or as a reflex test after atypical cytology results is now gaining wider acceptance. The importance of full genotyping and viral load determination has been demonstrated to enhance the clinical understanding of the viral infection progression during follow-up or after treatment, thereby providing clinicians with supplementary tools for optimized patient management. We developed a new analysis method for the RIATOL quantitative PCR assay, and validated and implemented it in the laboratory of clinical molecular pathology at Algemeen Medisch Laboratorium (AML), under national accreditation and following the International Organization for Standardization guidelines. This study presents the successful validation of a high-throughput, multitarget HPV analysis method, with enhanced accuracy on both qualitative and quantitative end results. This is achieved by software standardization and automation of PCR curve analysis and interpretation, using data science and artificial intelligence. Moreover, the user-centric functionality of the platform was demonstrated to enhance both staff training and routine analysis workflows, thereby saving time and laboratory personnel resources. Overall, the integration of the FastFinder plugin semi-automatic analysis algorithm with the RIATOL real-time quantitative PCR assay proved to be a remarkable advancement in high-throughput HPV quantification, with demonstrated capability to provide highly accurate clinical-grade results and to reduce manual variability and analysis time. Key Points•FastFinder algorithm uses data science and artificial intelligence methods on PCR metrics for robust Cq determination, enhancing curve interpretation accuracy.•Quantitative validation of >1000 clinically collected cervical samples revealed improved human papillomavirus load accuracy, potentially enabling patient triage strategies in screening and disease follow-up.•Use of software automation in a high-throughput molecular diagnostics laboratory saves time and resources, making it a comprehensive solution for large-scale PCR testing.•Cloud-based system provides a secure and scalable solution with user-centric design and traceability built and designed under an International Organization for Standardization 13485–certified quality management system. •FastFinder algorithm uses data science and artificial intelligence methods on PCR metrics for robust Cq determination, enhancing curve interpretation accuracy.•Quantitative validation of >1000 clinically collected cervical samples revealed improved human papillomavirus load accuracy, potentially enabling patient triage strategies in screening and disease follow-up.•Use of software automation in a high-throughput molecular diagnostics laboratory saves time and resources, making it a comprehensive solution for large-scale PCR testing.•Cloud-based system provides a secure and scalable solution with user-centric design and traceability built and designed under an International Organization for Standardization 13485–certified quality management system. Although valuable cytology screening methods were established long ago and are still widely used for detecting precancerous lesions of the cervix, cervical cancer is still today the fourth most frequent malignant tumor diagnosed in women worldwide.1Sung H. Ferlay J. Siegel R.L. Laversanne M. Soerjomataram I. Jemal A. Bray F. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.CA Cancer J Clin. 2021; 71: 209-249Google Scholar The association of persistent human papillomavirus (HPV) oncoprotein expression in infected basal cells is now well recognized as the most significant etiological factor in virtually all cervical cancer cases.2Schiffman M. Castle P.E. Jeronimo J. Rodriguez A.C. Wacholder S. Human papillomavirus and cervical cancer.Lancet. 2007; 370: 890-907Google Scholar,3Walboomers J.M.M. Jacobs M.V. Manos M.M. Bosch F.X. Kummer J.A. Shah K.V. Snijders P.J.F. Peto J. Meijer C.J.L.M. Munoz N. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide.J Pathol. 1999; 189: 12-19Google Scholar More than 40 HPV genotypes can infect the anogenital area,4Barzon L. Militello V. Pagni S. Franchin E. Dal Bello F. Mengoli C. Palù G. Distribution of human papillomavirus types in the anogenital tract of females and males.J Med Virol. 2010; 82: 1424-1430Google Scholar potentially resulting in a large spectrum of diseases (ie, low-risk HPV infections are often associated with benign lesions, whereas high-risk HPV genotypes are recognized by the International Agency for Research on Cancer as class I/IIa carcinogens).5World Health OrganizationIARC Monographs on the Evaluation of Carcinogenic Risks to Humans: Volume 90 Human Papillomaviruses. Vol. 90. International Agency for Research on Cancer, Lyon, France2007Google Scholar Likewise, it has been clinically demonstrated that HPV-based screening methods lead to a significantly increased testing sensitivity, particularly for persistent low viral infections, therefore decreasing the incidence of high-grade cervical intraepithelial neoplasia (CIN2+) and cancer, when compared with cytology-based methods.6Koliopoulos G. Nyaga V.N. Santesso N. Bryant A. Martin-Hirsch P.P. Mustafa R.A. Schünemann H. Paraskevaidis E. Arbyn M. Cytology versus HPV testing for cervical cancer screening in the general population.Cochrane Database Syst Rev. 2017; 8CD008587Google Scholar,7Ogilvie G.S. van Niekerk D. Krajden M. Smith L.W. Cook D. Gondara L. Ceballos K. Quinlan D. Lee M. Martin R.E. Gentile L. Peacock S. Stuart G.C.E. Franco E.L. Coldman A.J. Effect of screening with primary cervical HPV testing vs cytology testing on high-grade cervical intraepithelial neoplasia at 48 months.JAMA. 2018; 320: 43Google Scholar With a clear focus toward the large-scale implementation of primary HPV screening worldwide, in 2020, 425 HPV test variants were applied for diagnostic purposes, yet the vast majority did not officially publish any analytical and/or clinical evaluation data that ensure the fulfillment of the safety requirements for the use in clinical settings.8Arbyn M. Snijders P.J.F. Meijer C.J.L.M. Berkhof J. Cuschieri K. Kocjan B.J. Poljak M. Which high-risk HPV assays fulfil criteria for use in primary cervical cancer screening?.Clin Microbiol Infect. 2015; 21: 817-826Google Scholar,9Poljak M. Oštrbenk Valenčak A. Gimpelj Domjanič G. Xu L. Arbyn M. Commercially available molecular tests for human papillomaviruses: a global overview.Clin Microbiol Infect. 2020; 26: 1144-1150Google Scholar Henceforth, the international scientific community has been strongly advocating for compliance with population-based screening HPV assays, taking into consideration an optimal balance between clinical sensitivity and specificity for CIN2+ detection, along with interlaboratory reproducibility.10Meijer C.J.L.M. Berkhof J. Castle P.E. Hesselink A.T. Franco E.L. Ronco G. Arbyn M. Bosch F.X. Cuzick J. Dillner J. Heideman D.A.M. Snijders P.J.F. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older.Int J Cancer. 2009; 124: 516-520Google Scholar Although most of the commercially available HPV assays have limited genotyping capacity, full genotyping assays still play a pivotal role in risk management of persistent infections and/or the follow-up of residual viral loads after treatment of cervical lesions, as well as provide unique epidemiologic data for monitoring of vaccination effects in the population. In addition, other HPV-related factors, such as viral load, infections with multiple types, or viral as well as host methylation status, have been associated with high-grade disease.11Woodman C.B.J. Collins S.I. Young L.S. The natural history of cervical HPV infection: unresolved issues.Nat Rev Cancer. 2007; 7: 11-22Google Scholar,12Adcock R. Cuzick J. Hunt W.C. McDonald R.M. Wheeler C.M. Joste N.E. Kinney W. Wheeler C.M. Hunt W.C. McDonald R.M. Robertson M. Waxman A. Jenison S. Gage J.C. Castle P.E. Benard V. Saslow D. Kim J.J. Stoler M.H. Cuzick J. Pressley G.R. English K. Role of HPV genotype, multiple infections, and viral load on the risk of high-grade cervical neoplasia.Cancer Epidemiol Biomarkers Prev. 2019; 28: 1816-1824Google Scholar Although high viral loads have been shown to be important, especially for α-9 HPV-type infections,13Fu Xi L. Schiffman M. Ke Y. Hughes J.P. Galloway D.A. He Z. Hulbert A. Winer R.L. Koutsky L.A. Kiviat N.B. Type-dependent association between risk of cervical intraepithelial neoplasia and viral load of oncogenic human papillomavirus types other than types 16 and 18.Int J Cancer. 2017; 140: 1747-1756Google Scholar viral latency and regression of cervical lesions still remain an analytical challenge. HPV-based screening technology is shifting toward higher data resolutions, enabling more comprehensive and detailed interpretation of disease progression. Yet, the increase of data volume simultaneously poses new challenges regarding timely interpretation and manual errors. To address this issue, the integration of data science and artificial intelligence (DS/AI) is gradually proving its potential in the clinical diagnostic setting. Once an algorithm and its parameters are established and are validated to perform on par or better than a traditional approach, the benefit of applying DS/AI is not merely limited to accuracy and reproducibility but also allows laboratories to optimize turnaround times and amount of manual review time as well. In this study, the development of a tailored HPV multiwell, multitarget, complex quantitative model for PCR assay interpretation of results is described, using a training set of clinical validated samples to confirm assay accuracy and overall performance and consecutively deploying the model in a routine setting. Annually, >120,000 liquid-based cytology samples are received and processed by the laboratory of clinical and molecular pathology at A.M.L. (Sonic Healthcare, Antwerp, Belgium) as a national HPV reference laboratory for cervical cancer screening and diagnostic follow-up. DNA extraction and amplification protocols are performed as previously described14Micalessi I.M. Boulet G.A. Bogers J.J. Benoy I.H. Depuydt C.E. High-throughput detection, genotyping and quantification of the human papillomavirus using real-time PCR.Clin Chem Lab Med. 2012; 50: 655-661Google Scholar and are fully clinically validated for CIN2+ sensitivity and specificity.15Benoy I. Xu L. Vanden Broeck D. Poljak M. Oštrbenk Valenčak A. Arbyn M. Bogers J. Using the VALGENT-3 framework to assess the clinical and analytical performance of the RIATOL qPCR HPV genotyping assay.J Clin Virol. 2019; 120: 57-62Google Scholar,16Depuydt C.E. Benoy I.H. Beert J.F.A. Criel A.M. Bogers J.J. Arbyn M. Clinical validation of a type-specific real-time quantitative human papillomavirus PCR against the performance of hybrid capture 2 for the purpose of cervical cancer screening.J Clin Microbiol. 2012; 50: 4073-4077Google Scholar Briefly, the RIATOL TaqMan-based quantitative real-time PCR (qPCR) HPV genotyping assay is performed at ultralow reaction volume (6 μL) on a LightCycler 480 type I instrument (Roche Diagnostics, Basel, Switzerland). For each sample and control, eight multiplex reactions are analyzed, overall targeting 18 unique HPV types: HPV6E6, 11E6, 16E7, 18E7, 31E6, 33E6, 35E6, 39E7, 45E7, 51E7, 52E7, 53E6, 56E7, 58E7, 59E7, 66E6, 67L1, and 68E7 (Table 1). β-Globin expression is used as both internal cellular quality control and normalizer of viral copies detected for HPV-positive samples.17Steinau M. Rajeevan M.S. Unger E.R. DNA and RNA references for qRT-PCR assays in exfoliated cervical cells.J Mol Diagn. 2006; 8: 113-118Google Scholar Sample DNA and virus concentration are calculated from standard curves generated by dilution series of HPV-specific synthetic constructs (gBlocks Gene Fragments; Integrated DNA Technologies, Coralville, IA). Viral concentration, expressed as mass concentration, is used to calculate the viral load of each specific HPV type, expressed in copies per volume of extracted DNA, using the following equations:Numberofcells=DNAconcentrationng/μLweighthumangenomeng/cellcells/μL(1) Viralload=HPVconcentrationcopies/μLNr.cellscells/μLcopies/cell(2) Table 1Overview of the RIATOL Multiplex PCR Genotyping Assay Design (Type-Specific Exon Target and Fluorophore Probe Associated) Categorized by Cancer Risk StratificationClinical associationHPV genotypeMixFluorescent dyeHigh-risk HPVHPV 16 (E7)2FAMHPV 18 (E7)7FAMHPV 45 (E7)3FAMHPV 33 (E6)4VICHPV 58 (E7)4Cy5HPV 31 (E6)2VICHPV 52 (E7)4FAMHPV 35 (E6)5Cy5HPV 59 (E7)5FAMHPV 39 (E7)1FAMHPV 51 (E7)6FAMHPV 56 (E7)3VICHPV 68 (E7)3Cy5Potential high-risk HPVHPV 66 (E6)6VICHPV 53 (E6)6Cy5HPV 67 (L1)1VICLow-risk HPVHPV 6 (E6)5VICHPV 11 (E6)8VICCY5, cyanine 5; E, exon; FAM, fluorescein amidites; HPV, human papillomavirus; VIC, 2′-chloro-7′-phenyl-1,4-dichloro-6-carboxyfluorescein. Open table in a new tab CY5, cyanine 5; E, exon; FAM, fluorescein amidites; HPV, human papillomavirus; VIC, 2′-chloro-7′-phenyl-1,4-dichloro-6-carboxyfluorescein. The analytical PCR sensitivity of the different type-specific HPV assays varies between 1 and 100 viral copies per reaction,16Depuydt C.E. Benoy I.H. Beert J.F.A. Criel A.M. Bogers J.J. Arbyn M. Clinical validation of a type-specific real-time quantitative human papillomavirus PCR against the performance of hybrid capture 2 for the purpose of cervical cancer screening.J Clin Microbiol. 2012; 50: 4073-4077Google Scholar,18Depuydt C.E. Boulet G.A. Horvath C.A.J. Benoy I.H. Vereecken A.J. Bogers J.J. Comparison of MY09/11 consensus PCR and type-specific PCRs in the detection of oncogenic HPV types.J Cell Mol Med. 2007; 11: 881-891Google Scholar and the threshold of positivity is set to log10 of 6.493 copies/mL of sample.15Benoy I. Xu L. Vanden Broeck D. Poljak M. Oštrbenk Valenčak A. Arbyn M. Bogers J. Using the VALGENT-3 framework to assess the clinical and analytical performance of the RIATOL qPCR HPV genotyping assay.J Clin Virol. 2019; 120: 57-62Google Scholar At the end of each amplification run, the PCR run file is either analyzed manually in the LightCycler 480 Software (Roche Diagnostics) by trained operators (reference method) or exported via Microsoft (Redmond, WA) Azure cloud to the FastFinder online platform for semi-automated analysis and authorization of results. The core of the FastFinder software is the FastFinder Analysis module (FastFinder Analysis version 4.7.4; Velsera, Charlestown, MA), which contains the execution and visualization framework. The assay-specific interpretation information, an algorithm, and decision mechanism to go from raw PCR curve data to final sample result are contained with an assay plugin, which is the analysis software component tied to a particular assay. FastFinder is a commercially available software that can be customized for various PCR assays via the use of assay plugins. FastFinder assay plugins are versioned separately from the core platform to allow for consistent data analysis across consecutive versions of the analysis software, where every new version of the software is revalidated to ensure continuity in the validated state. The plugin software and the core platform software can easily, quickly, and separately evolve to meet the changing needs in a deployment at scale, by enabling the user interface, systems integrations, dashboard visualizations, and external interactions to be updated separately from the assays. The FastFinder platform has been developed within an International Organization for Standardization 13485–certified quality management system. The data are hosted on Microsoft Azure cloud service. An assay-specific plugin (HPV Geno/Quant AML, GUID: c352ed8c-ba71-4d26-9a98-cf3a88f5e6de) was initially designed through an AI-based results-driven fine-tuning process based on analytical cycle threshold cutoffs (Cq cutoffs), baseline corrections, and cross- and/or prevalence-driven contamination detection. This information is sent to the algorithm along with the raw PCR data, ensuring that the algorithm has access to the precise experimental conditions and parameters necessary for accurate analysis (Figure 1). The algorithm for qPCR curve calling must fulfill three objectives while going from raw qPCR data to a call. First, the raw data need to be baseline corrected to make curves comparable. Next, a Cq value has to be determined for each curve; and finally, each curve also receives a call (positive or negative) or can be flagged for manual interpretation. The algorithm is designed to be a generic and an out-of-the-box solution, independent of the assay or qPCR thermocycler from which the data originates. The HPV Geno/Quant AML assay plugin contains the default algorithm parameters as the data fall within the expected boundaries for which the generic approach provides the expected outcome. For assays with data falling outside the expected boundaries, custom assay-specific parameters can tune the results of the algorithm to better fit the expected outcome, making FastFinder a comprehensive solution across assay designs, thermocycler instruments, and PCR technologies. After the FastFinder analysis baseline correction implementation compensates for the arbitrary fluorescence offset of each curve, any additive linear background signal with a technical or assay design–related origin and sample handling–related anomalies are accounted for as well. In this process, each curve is characterized by a set of metrics functionally related to the qPCR method, and conclusions from other curves from the same target in the same run can be drawn. The call (positive or negative) for each curve is determined in a domain knowledge–driven model (expert AI) aggregating curve characteristics computed earlier, including the presence and position of qPCR phases, the Cq values, and fluorescence amplitudes of all curves of the same target within the run. This expert AI model consists of complex decision trees, reflecting human decision logic used in manual analysis. Curves with conflicting metrics (technical), those with anomalies (functional), or those identified as late positive amplification are flagged for manual interpretation, enabling instant notification of quality issues within the laboratory workflow and therefore allowing for immediate technical intervention. Next, the algorithm output is used as input for the decision tree that provides the final sample result. The decision tree contains a set of rules that account for every possible outcome. Individual target calls are aggregated to generate the valid sample result: i) the negative control of each run must be negative for all targets; ii) every target-specific positive control must be individually detected and may not deviate more than 3 SDs from the average Cq value determined (Figure 2A); and iii) the internal cellular control gene (β-globin) must be detected at a concentration of at least 0.12 ng/μL per sample. If these criteria are not met, the results might be invalid and hence require manual intervention, possibly triggering a full rerun for each sample, including extraction and PCR amplification. Additionally, the software will provide specific error messages the user has to review before an analysis can be authorized for sign-out (Figure 2B). Specifically, a message is dispatched when the Cq is abnormally low (Cq 0.80 was considered as almost perfect. The quantitative differences between the two methods were evaluated on a Cq level by regression statistics applied on the internal control gene calling, as well as Bland-Altman analysis for all PCR-detected targets. The extent of agreement was assessed by plotting the mean difference between both methods for each specimen, with the respective calculated 95% CI of the limits of agreement. Limit of detection and limit of quantification for each target using the new FastFinder software were recalculated on the basis of the SD of the concentration (σ) and the slope (S) of the calibration curve, according to the international committee on harmonization,20European Medicines Agency: ICH Q2(R2) Guideline on Validation of Analytical Procedures: Step 5. European Medicines Agency, Amsterdam, the Netherlands2023Google Scholar following the formulas:Limitofdetection=3.3σ/S,andlimitofquantification=10σ/S(3) The statistical significance of the viral load difference was determined for each HPV type using paired t-test analysis (P < 0.05; GraphPad Prism version 9.4.0). Turnaround-time efficiency regarding both the training and the analysis of results were calculated by comparing retrospective data from manual routine and prospective results of FastFinder implementation. Data from 10 users who were trained for manual analysis between 2017 and 2020 were compared against the FastFinder training data from six individuals who were trained between June and November of 2022. Manual analysis metrics were extrapolated from retrospective registries based on performance of full-time equivalent technicians retrieved from 4 months of routine analysis (n = 114). FastFinder-assisted analysis was assessed via the audit trail embedded in the platform over consecutive routine runs performed by the trained users (n = 63). A total of 1043 samples, containing HPV type–specific positivity ratio according to prevalence of each type in the screening population, were analyzed by both manual procedures using the LightCycler 480 Software and FastFinder assay plugin specifically developed for the RIATOL qPCR assay (HPV Geno/Quant AML). Of a total of 19,760 curves analyzed, only 46 curves (0.2%) showed qualitative discrepant results (detected/not detected) between the manual and the FastFinder-assisted analysis, resulting in a κ coefficient of agreement of 0.983 (95% CI, 0.978–0.988), interpreted as almost perfect agreement between the two categorical variables.21Landis J.R. Koch G.G. The measurement of observer agreement for categorical data.Biometrics. 1977; 33: 159-174Google Scholar Moreover, further examination of the nonconcordant curves revealed the classification of reasons for discrepancy in six categories (Table 2), where human error during manual analysis accounted for most of the events (35 of 46 discrepant events were classified as human errors after revision). This shows that the prompts flagged by the software on curves with conflicting metrics or functional anomalies do assist the user with important decision-making information that otherwise would have been dismissed eg, false positivity due to cross-contamination, color compensation or false negativity due to low end-fluorescence values (relative fluorescence units). In the absence of this automated assistance, the error rate of the manual analysis was higher. Specifically, only two discordant curves (representing 4.35% of all discordant curves) were incorrectly called by the software as false negatives, and this was due to low end-fluorescence signals, likely from low efficiency of the PCR for potentially high-risk HPV types. In fact, the multiplexing of RIATOL qPCR assay is designed in a way that the HPV genotypes strongly associated with high-grade lesion progression (HPV 16, 18, and 45) are reported with a robust fluorophore probe fluorescein amidites (FAM) (Table 1), therefore reducing the possibility that analytical limitations of the software will have an impact.22Maver P.J. Poljak M. Primary HPV-based cervical cancer screening in Europe: implementation status, challenges, and future plans.Clin Microbiol Infect. 2020; 26: 579-583Google ScholarTable 2Coefficient of Agreement Statistics and Categorization of Reasons for End-Result Nonconcordance per CurveVariableValue%κ Coefficient of agreement0.983SEM of κ0.00295% CI0.978–0.988Observed agreements, N19,714Total observations17,015Total discrepancies46100FF right call4496FF wrong call24Weak positive sensitivity920Human error: cross-contamination817Human error: color compensation511Human error: low EF1124Human error1124Software error: low EF24EF, end fluorescence; FF, FastFinder. Open table in a new tab EF, end fluorescence; FF, FastFinder. For viral load measurement, real-time quantitative PCR has emerged as the preferred method because of its ability to offer type-specific assays, enabling normalization for cellular content within a dynamic range of at least six logs (106 to 100 copies). See target-specific calculation of limit of detection and limit of quantification using FastFinder software for curve interpretation in Supplemental Table S2. In this study, it was first confirmed that the HPV positivity prevalence rate on a cohort of 1040 Belgian women attending national cervical cancer screening program (high-risk HPV, n = 240, 23%) (Supplemental Table S1) is similar to the rate reported among other Western-European studies (Spain: https://hpvcentre.net/statistics/reports/ESP.pdf; France: https://hpvcentre.net/statistics/reports/FRA.pdf; Germany: https://hpvcentre.net/statistics/reports/DEU.pdf; all last accessed May 2023). Although it is well established that HPV-based screening considerably increases sensitivity compared with cytology-based screening, its specificity for detection of CIN2+ is typically significantly lower.6Koliopoulos G. Nyaga V.N. Santesso N. Bryant A. Martin-Hirsch P.P. Mustafa R.A. Schünem