Answers are grounded in this paper and cite their sources.
Mental disorders affect about 970 million people worldwide, yet few are ever screened in a clinic. Reddit posts contain people's own accounts of distress, making automated text classification an attractive, low-cost screening aid, but published results for competing transformer models use different datasets and protocols and cannot be compared. This study compared BERT, RoBERTa, MentalBERT, and DeBERTa-v3 under a single protocol to recommend a model for low-resource, multi-class screening. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was combined with a benchmark on Reddit posts labelled by subreddit into four classes: anxiety, depression, bipolar disorder and schizophrenia. Each model was fine-tuned with identical hyperparameters and class-weighted cross-entropy loss on a stratified sample of 20,001 posts on one NVIDIA T4 graphics processing unit (GPU), then tested on 3,001 held-out posts using macro-F1, accuracy and per-class precision and recall. Stability came from 30 bootstrap resamples of the test predictions, pairwise differences were tested with McNemar's test, and a decision matrix weighted macro-F1 (30%), domain pretraining relevance (25%), stability (15%), literature evidence (15%), efficiency (10%) and pretraining data overlap (5%). RoBERTa-base had the highest bootstrap macro-F1 (0.806), with DeBERTa-v3-base close behind (0.805). MentalBERT did not differ significantly from RoBERTa-base or BERT-base (p > .05). The matrix ranked MentalBERT first (8.07/10), but a sensitivity analysis showed that this depends on the judged criteria. Domain-adaptive pretraining gave no measurable advantage at this scale. Model choice should weigh accuracy, cost and domain fit together, and outputs should serve only as a triage signal.
No takes yet. Share an insight, caveat, or question.
Muturi et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: