A unified comparative evaluation of machine learning, deep learning and GPT-2 for suicide ideation detection from social media

Abstract Social media platforms have progressively transformed into venues for individuals to express mental anguish and suicidal thoughts. This research examines various computer algorithms for detecting suicide-related content in social media text within a standardized experimental framework. Conventional machine learning models, including naive Bayes, logistic regression, random forest, support vector machine, gradient boosting machine, LightGBM, and XGBoost, were assessed utilizing Universal Sentence Encoder (USE) and Sentence-BERT embeddings. Additionally, deep learning architectures such as CNN, LSTM, and RNN were examined alongside a refined GPT-2 transformer model for binary classification. The models undergo training and evaluation under identical preprocessing and assessment conditions to ensure a fair comparison. Moreover, we conducted additional tests on a selection of the highest-performing models utilizing multiple random seeds and stratified cross-validation to enhance robustness and assess repeatability. Among the evaluated approaches, GPT-2 achieved the highest overall performance, attaining an accuracy of 98.25% along with balanced precision and recall for both suicidal and non-suicidal categories. CNN exhibited superior performance among deep learning baselines, but USE embeddings generally surpassed Sentence-BERT in the majority of standard machine learning models. The findings indicate that transformer-based language models can deliver superior contextual classification performance on the assessed dataset. The findings are limited to a binary classification dataset derived from Reddit and require more validation across different platforms, languages, and populations.