TE-SFMTL: enhanced multi-task learning framework with triple-encoder for extremely imbalanced Arabic toxicity detection

Multi-task learning (MTL) improves hate speech detection by sharing knowledge across related tasks, but standard joint training fails under extreme class imbalance (>100:1). For morphologically rich, low-resource languages like Arabic, where minority classes may have fewer than 100 training examples, this challenge is particularly pronounced. Additionally, dialectal variation complicates the process of feature learning. This paper presents Two-Stage TE-SFMTL (Staged Focal Multi-Task Learning), a gradient isolation framework addressing extreme imbalance through strategic task sequencing. Stage 1 co-trains moderately imbalanced tasks (offensive, hate speech, vulgarity, sentiment) to build shared representations. Stage 2 freezes the encoder and all Stage 1 heads, training only the extremely imbalanced task head in complete gradient isolation, eliminating inter-task gradient competition that causes rare-class failure during joint training. Two-stage TE-SFMTL is evaluated on three Arabic benchmarks: OSACT2022, OSACT2020, and L-HSAB. On OSACT2022’s violence task (only \(~ \) 60 training positives, IR 141:1), the framework recovers from near-chance Stage 1 performance (AUROC 0.516) to strong detection (Stage 2 AUROC \(0.819 ± 0.021\) across three seeds, peak 0.844), a 58.7% relative improvement. Without the two-stage training, the violence AUROC remains at 0.598, which confirms that gradient isolation provides a +37.0% relative gain. Cross-dataset validation confirms generalizability: on OSACT2020, hate speech AUROC improves from 0.348 to 0.950 (+173.0%), achieving 85.4% average F1; on L-HSAB, it improves from 0.623 to 0.958 (+53.9%), achieving 81.6% average F1. The approach minimizes annotation requirements in low-resource scenarios, allowing for effective detection of rare classes with as few as 60 positive examples. Ablation studies indicate that the two-stage gradient isolation is the single most impactful component; without it, violence AUROC drops from 0.844 to 0.598 ( \(-29.2%\) ). This demonstrates that addressing extreme imbalance requires architectural solutions beyond loss-function modifications.