Answers are grounded in this paper and cite their sources.
Machine-generated-text detection has expanded across languages, domains and generators, but Romanized code-mixed varieties remain weakly represented. We construct BanglishAIGT v1, an 8,100-text experimental benchmark of Banglish (Romanized Bengali mixed with English) containing 3,067 human e-commerce reviews and 5,033 direct generations from three commercial LLMs. We compare four word/character TF-IDF classifiers with mBERT and XLM-R on fixed generator-stratified splits. Clean-test performance is near ceiling: the validation-selected Linear SVM reaches 0.988 macro-F1 and XLM-R 0.991, and the two are statistically indistinguishable in a paired analysis. Removing capitalization, punctuation and emoji at test time costs Linear SVM 1.3 macro-F1 points but costs mBERT 17.6 and XLM-R 14.8. Removing AI e-commerce examples from training lowers Linear SVM macro-F1 on e-commerce text from 0.978 to 0.864, whereas all six models retrained within the e-commerce domain alone still reach 0.960–0.983. For mBERT, SHAP attributions to commas and periods on 250 randomly sampled texts remain positive under resampling, although the wider token ranking is unstable. Together, these results give converging, non-causal evidence that near-perfect matched-split accuracy partly reflects formatting, domain and generator cues. We present the benchmark as a case study in why accuracy should be reported together with uncertainty, perturbation, restricted training and attribution, not as a detector ready for deployment.
No takes yet. Share an insight, caveat, or question.
Hira et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: