Data requirements for suicidal and depressive thought detection from Reddit™ with Recurrent Neural Networks

With suicide and depression rates becoming a growing concern worldwide, timely identification of individuals at risk is important for providing support and intervention. Identifying these patterns plays a critical role in detection, however distinguishing between the two mental illnesses remains a challenging task because afflicted individuals’ texts and/or messages at online discussion forums express similar verbiage of hopelessness and emotional struggle. Machine learning models offer a solution to this conundrum by utilizing labeled data to analyze language patterns and categorize text while accounting for these subtle variations between the two mental states. However, the process of collecting and managing large datasets for training these models can be resource-intensive and costly. This study’s objective was to determine if there was an ‘ideal’ amount of data required to train models effectively, balancing optimal performance with resource efficiency. By evaluating the effect of different data sample sizes, this research provides a practical insight for developing scalable and cost-effective models. Recurrent Neural Network (RNN)-based architectures, including the Simple RNN model, Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs) were employed to detect suicidal thoughts in Reddit™ posts and distinguish them from depressive content. Network complexity was also examined by experimenting with different node configurations within the models. Model performance, across all models, remained stable and effective when trained on dataset sizes ranging from 36,000 (20% of the data) to 126,000 samples (70% of the data), with minimal improvements observed beyond this range, despite the full dataset containing 180,000 samples. Soberingly, none of the models could accurately identify ‘normal’ texts from a manually generated post-hoc sample which mimicked teen slang, demonstrating that – regardless of the number of posts the models trained on – they were ultimately not able to decipher the contextual and semantical vernacular that is representative of texts on social media posts from this vulnerable demographic.