Digital hate annotation constitutes an important task for training automatic classification algorithms that are robust across country contexts yet also sensitive to their unique conditions. However, perceptual heterogeneity in how hateful content is evaluated introduces a great challenge for consistent rating and labeling, as interpretations of norm violations are shaped by individual and contextual factors. In this study, we conduct a 2 (uncivil vs. intolerant content) × 2 (ingroup vs. outgroup targeting) × 2 (annotator vs. user role) mixed online experiment in Austria, France, Hungary, and Sweden ( N = 1,718), in which participants rate and label social media comments. Intolerance is consistently perceived as more severe and labeled more often as hateful compared to incivility across all countries. However, we find mixed support for differences between ingroup and outgroup targeting, only limited cross‐country evidence that explicitly assigning participants an annotator role influences their severity ratings, and no evidence that it affects their labeling decisions compared to when they act as regular users. While the influence of sociodemographics on perception is highly dependent on the country context, we find migration attitudes to be a country‐invariant predictor.
Kirchmair et al. (Thu,) studied this question.