843 Background: Circulating tumor DNA (ctDNA) in patients diagnosed with cancer has emerged as a critical biomarker to detect and monitor molecular residual disease for timely decision-making. Currently, medical providers rely on clinical reports that require manual data curation to understand a patient’s medical history and make informed treatment recommendations. Here, we implemented a novel approach that automates clinical data abstraction, combined with a user-friendly interface, resulting in an interactive overview plot, readily available to medical providers for critical decision-making. Methods: To evaluate the performance of Natera’s AI-assisted automated abstraction process, we developed a data dictionary specific to gastrointestinal cancers that covered 13 event types across six categories: Patient Information, Condition, Procedure, Biomarker, Outcome, and Clinical Follow-up. In a reference cohort of 50 patients, datasets generated from two approaches were compared: (1) fully manual abstraction by clinical abstractors, and (2) AI-automated extraction using a large language model (LLM) pipeline followed by human validation through a customized interface. Patient-level variables (e.g., tumor location, cancer stage, morphology, treatment regimens, type and date of surgery) were compared. Discordant cases underwent additional in-depth consensus review by clinical abstractors to adjudicate the results. Results: Reference cohort comprised of predominantly colorectal cancer (72%) with a smaller representation of other cancer types (eg. 6% appendiceal, 4% small intestine, 2% each of esophageal, anal, liver), stage II-III in 62% of cases, 44% female, median age 64.9 years. The fully manually abstracted dataset included 863 events, and the AI-assisted included 883 events. The AI-assisted abstraction demonstrated high concordance with manual abstraction across all selected fields (average 90% before review). Importantly, in many of the discordant cases, the AI-assisted pipeline provided more complete or accurate information than manual abstraction. For example, primary tumor location was captured more accurately with AI in 83% (5/6) of discordant cases, group staging in 75% (3/4) of discordant cases, and MSI status in 63% (5/8). Overall, key data elements showed excellent agreement, with the AI-assisted approach showing similar or superior accuracy for such fields as cancer type (98%), surgery date (94%), relapse date (92%), status of adjuvant and neoadjuvant therapy (96% and 100%). Conclusions: The AI-assisted abstraction platform, leveraging an LLM pipeline to present extractions to a clinical reviewer, demonstrates high concordance with manual curation and offers substantial improvements in abstraction throughput. This synergy of AI-driven extractions and expert oversight presents a highly effective and robust methodology for clinical data abstraction.
Grothey et al. (Sat,) studied this question.