Answers are grounded in this paper and cite their sources.
Artificial intelligence (AI), particularly machine-learning models, is increasingly applied to complex forensic chemical, material, and biological evidence. High reported performance does not itself demonstrate generalization to new forensic material. We conducted a systematic methodological review of 446 reports and 1,005 eligible evaluation regimes, examining independence of final evaluation, physical and biological parent grouping, information leakage, external or shifted testing, and transportability. Following source-based methodological assessment and human adjudication, source-supported leakage was classified in 284/446 reports (63.7%); 36/446 (8.1%) showed strong potential concern, 85/446 (19.1%) had uncertain boundaries, and 41/446 (9.2%) had no identified leakage concern. External or shifted evaluation was reported in 278/446 reports (62.3%), although only 94/278 (33.8%) had an evaluation satisfying five strict criteria for transportability of an unchanged pipeline. At regime level, 135/521 external or shifted evaluations (25.9%) satisfied these criteria. Source-supported leakage occurred in 195/308 Biological reports (63.3%) and 87/133 Chemical/Material reports (65.4%): prevalence ratio 1.033 (95% CI 0.889–1.200). The primary comparison did not identify a significant difference across these domains. Casework claims should therefore be evaluated against the independence of the relevant forensic unit, the complete development–test boundary, the shift actually examined, any target-data adaptation, and the applicable performance denominator.
No takes yet. Share an insight, caveat, or question.
Bailo et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: