Advancing Mental Health Detection Through Transfer Learning and Feature Fusion: Mitigating Data Imbalance in Large Transformer Models

This study addresses the challenges of poorly annotated data and class imbalance in mental health detection from social media. We propose an integrated approach combining weak classifiers with gradient boosting, leveraging LSTM, pretrained Transformer models (e.g., BERT, RoBERTa), and the MAMBA framework to analyze negative emotional states such as anxiety, depression, and anorexia. Data resampling and augmentation techniques effectively mitigated class imbalance, improving RoBERTa’s accuracy, recall, and F1-score from 0.792, 0.812, and 0.816 to 0.824, 0.856, and 0.857, respectively. A novel multi-source feature fusion method integrating local, global, and TF-IDF representations further enhanced performance, raising BERT’s accuracy, recall, and F1-score from 0.748, 0.760, and 0.766 to 0.785, 0.807, and 0.810. Experimental analysis using confusion matrices and error patterns revealed model behaviors and limitations in processing complex emotional data, offering a robust framework for advancing mental health detection systems.