Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
October 3, 2024BMC Medical Informatics and Decision MakingOpen Access

Validation of large language models for detecting pathologic complete response in breast cancer using population-based pathology reports

View Full Paper
Ask AI
Bookmark
Share

Authors

KCKen CheligeerGWGuosong WuALAlison Laws

Discussion

Loading...

Member takes

Overview

Retrospective cohort study demonstrates high accuracy of large language models detecting pathologic complete response in breast cancer, highlighting their potential for clinical surveillance.

Key Points

  • To evaluate the ability of open-source large language models to identify pathologic complete response within narrative breast cancer pathology reports.
  • Evaluated pathology reports from N=351 female breast cancer patients who underwent neoadjuvant chemotherapy and surgery between 2010 and 2017 in Calgary.
  • Extracted text embeddings using 15 transformer models classified with logistic regression, and fine-tuned a GPT-2 model connected to a feed-forward neural network within a secure health computing environment.
  • The optimized large language model pipeline achieved a sensitivity of 95.3% (95% CI: 84.0–100.0%) and a positive predictive value of 90.9% (95% CI: 76.5–100.0%).
  • The fine-tuned model achieved an overall F1 score of 93.0% (95% CI: 83.7–100.0%), outperforming conventional machine learning classification models.

Cite This Study

Cheligeer et al. (2024) studied this question.

synapsesocial.com/papers/6a0fb31d90ecb39bf65fac48https://doi.org/10.1186/s12911-024-02677-y
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Validating Large Language Models for Identifying Pathologic Complete Responses After Neoadjuvant Chemotherapy for Breast Cancer Using a Population-Based Pathologic Report Data2024 · 1 citations
  2. 2Performance of large language models for extracting clinical data from breast cancer pathology reports: a systematic review2026 · 3 citations
  3. 3Agent-based large language model system for extracting structured data from breast cancer synoptic reports: a dual-validation study2026 · 4 citations
  4. 4SCRIPT: Stratified clinical risk prediction from pathology reports using large language models2026
  5. 5Extraction and classification of structured data from unstructured hepatobiliary pathology reports using large language models: a feasibility study compared with rules-based natural language processing2024 · 9 citations