Abstract This paper presents a systematic review and meta-analysis of 62 studies that developed speech- and language-based AI for severe mental illnesses (SMI) (i. e. , characterized by substantial communication problems affecting speech production and language). We employed a random-effects meta-analysis using Restricted Maximum Likelihood (REML). We evaluated these studies using our proposed rigorous 16-item quality assessment framework, grouped into three domains: Study Design, Fairness and Explainability, and Transparency and Reproducibility. We conducted a meta-analysis of 45 studies (n = 4, 150) that reported accuracy and stratified them by ML type: deep learning (DL), shallow ML, and ensemble models. Our analysis showed that six DL-based studies (n = 340) achieved the highest pooled accuracy of 89. 7% (95% CI: 81. 4–94. 6; I ² = 73. 2%, ² = 0. 379). Twenty-one shallow ML studies (n = 1, 463) delivered a pooled accuracy of 85. 6% (95% CI: 82. 5–88. 2; I ² = 54. 6%, ² = 0. 120), while 18 ensemble model studies (n = 2, 347) achieved 83. 3% (95% CI: 77. 5–87. 8) with substantial heterogeneity. Across all studies, the pooled accuracy was 85. 1% (95% CI: 82. 3–87. 6; I ² = 76. 0%, ² = 0. 306). DL-based studies exhibited superior predictive performance despite smaller sample sizes, whereas shallow ML models delivered more consistent results across diverse datasets. These findings highlight the influence of model type and dataset characteristics on ML performance in SMI detection. R egression analyses across 45 studies, which showed no significant associations between the study quality domains—Study Design, Fairness and Explainability, and Transparency and Reproducibility—and test performance (p> 0. 45), indicating that these features need not be compromised to gain accuracy. These results indicate that variations in these methodological and reporting factors were not strongly linked to differences in predictive accuracy across studies. Overall, we observed that the included studies exhibited deficits in methodological rigour, transparency, reproducibility, and fairness, with many failing to report bias mitigation, preprocessing procedures, or code availability. Most studies were primarily research-oriented rather than clinically deployable, and significant heterogeneity in data collection tasks, extracted features, and ML pipelines further limited generalizability. We urge future ML studies for SMI to prioritize rigorous reporting of study design and methodology, while integrating clinically meaningful, interpretable, and fair AI practices. Such improvements can enhance the reliability, trustworthiness, and practical applicability of speech- and language-based AI for SMI diagnosis.
Parsapoor et al. (Mon,) studied this question.