Ranking Serious Maritime Occurrences with Machine Learning: Temporal Performance and Limits to Spatial Generalisation

Machine learning may support the prioritisation of reported maritime occurrences for expert review, but its usefulness depends on performance across time and geographical regions. This study assessed a leakage-controlled framework for ranking occurrences by their likelihood of being classified as “Serious” or “Very serious”. Extreme Gradient Boosting (XGBoost) was selected for its ability to model non-linear relationships and interactions among predictors in tabular occurrence data. The model was developed using 5815 European Maritime Safety Agency records with vessel, operational, temporal and geographical predictors, supplemented by severity-agnostic counts of preceding regional occurrences. Recorded consequence variables were excluded, and preprocessing parameters were estimated exclusively from the relevant training data. Evaluation comprised a held-out future-period test, temporal forward validation, spatial blocking and historical-context ablation. On the future-period test set, the area under the receiver operating characteristic curve (ROC-AUC) was 0.832, the area under the precision–recall curve (PR-AUC) was 0.620, and the balanced accuracy was 0.730. Sensitivity at the validation-selected threshold was 0.718. Among the highest-ranked 5% of occurrences, 79.5% were classified as “Serious” or “Very serious”. The mean ROC-AUC was 0.813 under temporal validation and 0.675 under spatial blocking; this descriptive contrast across different evaluation samples suggests limited geographical transferability under the evaluated settings. Historical-context ablation did not show an incremental discrimination benefit, while separate detection of “Very serious” occurrences remained unreliable at the fixed sentinel threshold. The results support further investigation of occurrence ranking for capacity-limited expert review. Operational use requires verification of predictor availability at the intended assessment time, improved calibration and external validation.