Machine learning detection of suicidal ideation in Persian language tweets using a large scale clinically annotated corpus

Suicide is a growing public-health concern in Iran and the broader Middle East, yet clinicians currently lack scalable tools for early detection of suicidal distress in Persian-speaking populations. Social media platforms offer a largely untapped opportunity for real-time surveillance, but research in Persian remains scarce due to the absence of clinically annotated datasets. To construct, to our knowledge, the first large-scale, expert-annotated Persian Twitter corpus for suicidal ideation and to evaluate whether machine-learning classifiers can reliably distinguish tweets expressing personal suicidal thoughts from non-suicidal content, including content that mentions suicide without personal intent. We retrieved Persian tweets posted between January 2020 and December 2021 via Twitter’s Academic Research API, using a psychiatrist-curated lexicon of 86 suicide-related and neutral query terms in two parallel retrieval streams of comparable size (balanced case-control design). After language verification, deduplication, and relevance filtering, tweets were annotated for the presence of personal suicidal ideation by two board-certified psychiatrists, following a protocol finalised in an independent double-annotation pilot phase (Cohen’s κ = 0.81 on the pilot subsample). The final corpus comprised 197,270 tweets (98,669 suicidal ideation, 98,601 non-suicidal). We trained linear support vector machine (SVM), logistic regression, and Bernoulli Naive Bayes classifiers on TF-IDF features (word 1–3-grams, 10,000 features) and evaluated them through seeded stratified 10-fold cross-validation, a keyword-masking ablation, a class-weighting sensitivity analysis, and probability-calibration analysis. The complete computational workflow is publicly released with a fixed random seed; exact numerical regeneration requires access to the restricted corpus. Logistic regression and linear SVM performed equivalently (F1 = 0.933 ± 0.001 and 0.933 ± 0.001; PR-AUC = 0.980 and 0.979), and both far exceeded a keyword-lexicon baseline, which achieved high precision (0.970) but missed 62% of suicidal tweets (recall = 0.378), showing that most expressions of suicidal ideation in the corpus do not contain explicit suicide-related query terms. When all 44 suicide-related query terms and their morphological variants were masked from test tweets, the SVM still achieved F1 = 0.898, indicating that the classifiers rely substantially on contextual linguistic cues beyond the lexicon terms. The logistic regression model was well calibrated (Brier score = 0.052; Cox calibration slope = 1.22), supporting the interpretability of its probability outputs. Supervised machine-learning models can reliably distinguish Persian tweets expressing personal suicidal ideation from everyday content, including in the majority of cases where no explicit suicide terminology is present. The annotated corpus, transparent methodology, and publicly inspectable, seeded pipeline presented here establish a dedicated framework for Persian-language suicide-risk research. These tools may support public-mental-health surveillance in Iran and across the Persian-speaking world, provided that deployment is embedded within human-oversight frameworks and appropriate ethical safeguards, and provided that performance is re-validated under realistic, low-prevalence conditions before any operational use.