Answers are grounded in this paper and cite their sources.
Toxic language detection for large language model safety has largely been framed as binary classification, yet unethical expressions differ in severity across types and human judgments of severity are far from uniform. We analyze a large-scale Korean ethics-verification corpus with eight type labels (seven unethical types plus neutral) and the individual intensity votes (0–3) of five raters per sentence (363,154 sentences; 1,001,030 votes). First, the seven unethical types form a three-tier perceived-severity hierarchy—censure (1.42) < discrimination, hate, and abuse (1.53–1.59) < sexual, violence and crime incitement (1.74–1.75)—supported by post-hoc tests: all between-tier differences are significant with substantial effect sizes (Holm-corrected p < 0.05; |Cliff’s δ| = 0.14–0.45), while within-tier effects are negligible. Second, automatically estimated sentiment valence barely explains the hierarchy (Spearman ρ = −0.142; AUC = 0.621; replicated with a Korean-specific sentiment model and with preceding dialogue context included in the input); the two most severe types show the most neutral surface valence, so sentiment signals capture best precisely the least severe type. Third, inter-rater agreement is low (ordinal Krippendorff’s α = 0.373), yet the disagreement is not explained by the basic demographics the corpus records—rater gender and age (equivalence tests bound the pooled gender effect at |d|≤0.013 and the age correlation at |ρ|≤0.002; unrecorded rater characteristics remain untested)—and instead grows with item severity (sentence-level ρ = 0.248). Harmfulness judgment appears to be a dimension largely independent of sentiment, whose annotation may benefit from continuous scales, multi-rater averaging, and severity-proportional rating resources.
No takes yet. Share an insight, caveat, or question.
Cheong et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: