Flagging urgent suspicion of lung cancer with AI CXR algorithms: optimising feature selection and performance

Objectives To investigate the impact of feature selection and thresholding on the performance of a commercial chest X-ray (CXR) algorithm for flagging urgent suspicion of lung cancer (USC) in a West of Scotland population. Materials and methods The datasets comprised 2887 prospectively collected CXRs from general practitioners and outpatient routine care in the West of Scotland. An additional validation dataset included 983 retrospectively collected CXRs. The index test was an algorithm probability score; the reference standard was USC, derived from a blinded expert radiologist panel review of CXR. Three feature selection approaches were compared: radiologist-selected features with commercial thresholding of individual features, radiologist-selected features with regression weighting for the combined score, and a model using Akaike Information Criterion (AIC) and LASSO regression approaches for feature selection. Performance was evaluated at two thresholds: the Youden index and 15% positive flag rate. Results Regression-based feature selection using the AIC and LASSO approaches resulted in a four-feature model (consolidation, nodule, tuberculosis and cavity) compared with the five features (cavity, nodule, mass, hilar enlargement and mediastinal widening) selected by clinical experts. Sensitivity was highest in the AIC/LASSO model, but was significantly impacted by a 15% flag rate threshold. Regression weighting for the combined score demonstrated superior performance compared to individual feature thresholds. Conclusion In this study, regression-based feature refinement against an expert-derived reference standard outperformed clinically intuitive selection. Commercial CXR algorithms require local calibration and task-specific optimisation for USC detection. Workflow and post-market deployment considerations can be addressed by constraining positive flag rates through a unified model threshold. Key Points Question How should users select and combine AI chest X-ray algorithm features to achieve optimal deployment performance for detecting urgent suspicion of cancer in routine practice? Findings Regression-based feature selection outperformed expert-selected features, and combining features under a single regression-weighted threshold outperformed individual feature thresholds for urgent cancer suspicion detection. Critical relevance statement Optimising chest X-ray AI for urgent cancer suspicion requires data-driven, regression-based feature selection rather than clinical intuition; health systems therefore need local capability to validate and calibrate algorithms for their population and task.