LLM-powered prostate cancer staging from PSMA-PET/CT reports using PROMISE v2

Accurate staging of prostate cancer is essential for guiding therapy and predicting outcomes. Prostate-specific membrane antigen (PSMA) PET/CT has become an established modality for disease assessment, and the PROMISE v2 framework provides standardized criteria for molecular imaging–based TNM (miTNM, molecular imaging TNM) classification. This study investigates the potential of large language models (LLMs) to automatically extract PROMISE v2 staging information from PSMA PET/CT reports. We retrospectively analyzed 1, 696 reports from first-diagnosis prostate cancer patients using the open-source Meta-Llama-3. 1-8B-Instruct (in the Q8₀ GGUF quantization) model, deployed securely within institutional infrastructure. Four prompting strategies were systematically compared: Zero-shot, advanced Zero-shot, few-shot, and chain-of-thought (CoT). Performance was evaluated using accuracy, precision, recall, and micro- and macro-averaged F1 scores. Advanced Zero-shot prompting achieved the highest performance for local staging (micro-F1 = 0. 65), while CoT prompting was superior for nodal (micro-F1 = 0. 79) and metastatic staging (micro-F1 = 0. 84). Error analyses revealed that Zero-shot approaches tended to collapse predictions into central categories, whereas advanced Zero-shot and CoT prompting yielded more balanced and stage-specific outputs. These findings demonstrate that LLMs can reliably map narrative PET/CT reports onto structured PROMISE v2 staging categories, with prompting strategy strongly influencing performance across staging dimensions. Our results highlight the feasibility of LLM-based workflows for standardizing prostate cancer staging, enabling reproducibility, and supporting large-scale outcome analyses.