Machine learning study demonstrates enhanced digital violence detection in Ecuadorian Spanish text, indicating the necessity of dialect-aware data curation and leakage-free evaluation.
Key Points
To develop and evaluate an inappropriate-content classification framework capable of accurately detecting digital violence while mitigating severe class imbalance and dialectal variation in Ecuadorian Spanish.
Extracted conversational threads using a window of three prior messages via a Selenium scraper, followed by human validation after zero-shot LLM labeling generated a 97.5% false alert rate.
Partitioned data into train, validation, and test splits (70/15/15) prior to augmenting minority classes using few-shot LLaMA 3.1 generation, semantic deduplication, and cosine-similarity filtering (threshold = 0.85).
Benchmarked BETO against mBERT across four experimental regimes, pairing the model with weighted loss functions to implement cost-sensitive learning.
BETO configured with semantic deduplication and weighted loss achieved 94.39% accuracy, a weighted F1-score of 0.9429, and a macro F1-score of 0.9022.
Naive oversampling triggered overfitting and artificial metric inflation, whereas controlled hybrid data generation significantly improved detection of underrepresented digital violence classes.