Abstract PS3-04-10: Development and Validation of a Living Decision Support Tool (Living-DST) in Oncology Using Agentic AI-Augmented Systematic Literature Review (SLR)

Key Result

The agentic AI-augmented Living-DST annotated 1,997 oncology trials 99% faster than humans with >90% accuracy, outperforming other AI tools in evidence completeness and precision.

Key Points

• The objective is to create a living decision support tool that aids oncologists in accessing updated clinical trial data effectively.
• Developed an agentic AI system to perform systematic literature reviews.
• Annotated 29,236 clinical trial abstracts from breast, lung, and prostate cancers.
• Evaluated the accuracy of the AI annotations against a validation set of 1,997 human annotations.
• Integrated structured data with national treatment guidelines to form a real-time decision support tool.
• Compared the tool's performance to other AI systems using treatment options in breast cancer.
• Achieved annotation accuracy of 95.1-97.2% for review variables.
• Extraction variable accuracy exceeded 90%, with half above 95%.
• Reduced AI annotation time to 7.25 hours compared to 727.4 hours required by humans, marking a 99% time savings.
• Outperformed other AI tools in completeness and accuracy of retrieved clinical evidence.