Pathogen prevalence, feature composition and cross-centre generalisability of machine learning diagnostic models for multi-pathogen respiratory infection

Objective: To evaluate the predictive information contained in the restricted surveillance feature set (demographic, temporal and specimen variables) for respiratory pathogen identification, and to examine factors associated with model performance. Methods: We retrospectively analysed 24,689 acute respiratory infection cases from seven sentinel hospitals in Sichuan, China; after quality control, 21,395 samples with complete 21-pathogen testing were included. Per-pathogen multi-label binary classifiers were trained using 15 features available at presentation; logistic regression, random forest, back-propagation neural network and XGBoost were compared under an identical split and threshold-selection protocol, with thresholds derived from out-of-fold probabilities. Label-definition sensitivity, leave-one-hospital-out cross-validation and prevalence-performance association were analysed. Leave-one-hospital-out validation gave Macro-F1 0.1073 ± 0.0289 (-30.9%) and Macro-AUC 0.6841 (-11.2%). Feature ablation showed temporal and non-temporal features were largely redundant (Macro-F1 0.1153 vs. 0.1147; AUC 0.6892/0.6865 vs. 0.7702). Treating untested as negative lowered Macro-AUC to 0.7653, whereas simulated false-negative label contamination had minimal effect. Conclusion: Models built on the restricted surveillance feature set showed limited diagnostic performance and were not demonstrated suitable for clinical deployment. Performance was strongly associated with pathogen prevalence and constrained by limited feature informativeness, while algorithm choice contributed little and cross-centre generalisability was limited. Priorities include multimodal feature integration, rare-pathogen accrual and prospective multicentre external validation.