Answers are grounded in this paper and cite their sources.
Extremist content spreads quickly on social media, so moderation needs classifiers that are accurate and whose confidence can be trusted. We study the Domain-Aware Transformer Ensemble (DATE): a BERTweet-base classifier (pre-trained on 850M English tweets), contextual-substitution and random-edit augmentation applied only inside each training fold, and a soft-voting ensemble of ten fold models. We evaluate on the Social Media Extremism Detection dataset (2,250 training posts, 749 labelled test posts) with a leakage-controlled protocol and report three kinds of evidence separately. (i) Cross-validation: single DATE models reach 0.822 accuracy (±0.027) on held-out folds. (ii) Held-out test posts: the ten-model ensemble reaches 0.843 accuracy (95% CI 0.814–0.868), macro-F1 0.744 and ROC-AUC 0.863, and its expected calibration error is 0.030 against 0.064 for single models (paired bootstrap difference \(-0.029\) , CI \(-0.046\) to \(-0.010\) ); over five training seeds, the ensemble is better calibrated than its members in every run (mean ECE 0.037 against 0.061). (iii) Competition: the submitted DATE ensemble scored 0.8507 on the private leaderboard of the Kaggle challenge ( \(≈\) 375 posts), tied for first place with two other teams among 164 teams; this margin is within sampling noise. Controlled ablations repeated over five seeds show that augmentation, class weights and the cosine schedule do not improve accuracy at full data size, although augmentation helps when training data is scarce; BERTweet beats RoBERTa-base in all five seeds (1.9 points on the test posts), and a TF-IDF baseline reaches comparable test accuracy. The study is limited to one English dataset whose annotation details (number of annotators, agreement) are not reported, and the test set has a different class balance from the training set.
No takes yet. Share an insight, caveat, or question.
Ali et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: