Key result
Llama3 models accurately extracted FFR/iFR values from angiography reports with 0.948 detection concordance and 0.906 exact numerical concordance, outperforming REGEX.
Why the study?
Extracting variables from unstructured coronary angiography and angioplasty reports is slow and hard to scale, limiting research and quality monitoring. The performance of open-source, locally deployable large language models on Portuguese cardiology reports was unknown.
Do open-source, locally deployable Large Language Models accurately extract FFR/iFR values and classify procedure types from unstructured Portuguese coronary angiography reports compared to a REGEX baseline?
Observational (n=1,314)
Do open-source, locally deployable Large Language Models accurately extract FFR/iFR values and classify procedure types from unstructured Portuguese coronary angiography reports compared to a REGEX baseline?
Answers are grounded in this paper and cite their sources.
Open-source LLMs, particularly Llama3, can accurately structure Portuguese coronary angiography reports and extract physiological indices, offering a practical, privacy-preserving solution to reduce manual data extraction for cardiovascular research and quality monitoring.
Introduction Coronary angiography and angioplasty reports contain essential anatomical, physiological, and procedural data for research, audits, and quality assessment. However, as unstructured text, extracting variables —such as lesion location, FFR/iFR values, and procedural details—is slow and hard to scale. This limits dataset size, delays research, and hinders quality monitoring. Although Large Language Models (LLMs) show promise in medical text processing, their performance on Portuguese cardiology reports is unknown. This study evaluates whether open-source, locally deployable LLMs can convert free-text angiography reports into structured data for research and quality improvement. Methods A total of 1,314 reports (2012–2023) were analyzed. Two tasks were assessed: extraction of FFR/iFR values per vessel and classification of procedure type. The reference standard was established through manual review by cardiology trainees. Llama3, Mistral and Medllama2 models were tested using zero-shot and one-shot prompting and compared with a REGEX baseline. Performance was measured using global concordance, sensitivity, positive predictive value (PPV) and F1-score, with strict exact-value matching for physiological indices. Results Of the 1,314 reports, 681 described angiography alone and 633 included angioplasty, totaling 591 FFR and 1,190 iFR values. MedLlama2 was excluded for inconsistent structured output. For procedure classification, REGEX achieved the highest global concordance (0.992) and PPV, while Llama3 one-shot showed the best sensitivity (0.998) and an F1-score of 0.984. For FFR/iFR detection , Llama3 zero-shot demonstrated the strongest performance, with global concordance of 0.948 and sensitivity of 0.872, surpassing both Mistral variants and REGEX. For exact numerical extraction, Llama3 one-shot reached the highest concordance (0.906). REGEX remained highly specific but less sensitive in detecting index presence, reflecting limitations with varied phrasing. Conclusion Open-source LLMs can accurately structure Portuguese coronary angiography reports, reducing the need for large-scale manual extraction. Llama3 models reliably detect FFR/iFR measurements and show high concordance in exact value extraction, outperforming REGEX and other open-weight models in linguistically complex tasks. This substantially decreases the number of reports requiring human review and enables dataset generation while preserving data privacy. Overall, LLM-based extraction emerges as a practical, deployable solution for research, quality monitoring, and audit, with clear potential to accelerate data-driven cardiovascular care.
No takes yet. Share an insight, caveat, or question.
Neves et al. (2026) conducted an observational in Patients undergoing coronary angiography and angioplasty (n=1,314). Open-source Large Language Models (Llama3, Mistral, Medllama2) with zero-shot and one-shot prompting vs. REGEX baseline and manual review by cardiology trainees was evaluated on Global concordance, sensitivity, PPV, and F1-score for extraction of FFR/iFR values and procedure classification. Llama3 models accurately extracted FFR/iFR values from angiography reports with 0.948 detection concordance and 0.906 exact numerical concordance, outperforming REGEX.
Synapse has enriched 10 related papers. Consider them for comparative context: