UOM at the NTCIR-18 RadNLP Task

The RadNLP 2024 (Natural Language Processing for Radiology) shared task at the international conference NTCIR-18 (English track) focuses on document classification for lung cancer staging, aiming to automatically determine the stage (i.e., the degree of progression) of lung cancer from radiology reports. Our approach involved data preprocessing, stratified data augmentation, and fine-tuning RadBERT—a transformer model pre-trained on radiology-specific text. We employed back-translation for data augmentation and 5-fold cross-validation to improve model robustness and address class imbalance. The results demonstrated that data augmentation significantly improved validation performance, with T accuracy increasing from 39.39% to 94.05% during K-fold validation and reaching 100% on the task validation set. However, a substantial performance gap was observed on the task test set, with joint accuracy dropping from 96.3% on the task validation set to 12.35%. This highlights challenges in model generalization due to limited dataset diversity and domain-specific language variability. This report details our methodology, results, and discusses the challenges encountered, highlighting the need for further research to improve the robustness and generalizability of automated lung cancer staging from limited radiology reports.