Answers are grounded in this paper and cite their sources.
Ethical refusals are essential for aligning large language models (LLMs) with safety constraints, yet users often penalize them in comparative evaluations. Whether this refusal penalty varies systematically across users remains unclear. Drawing on human–computer interaction research, we examine how users’ evaluative priorities and usage intensity shape responses to ethical refusals. Using the PRISM alignment dataset, which links LLM evaluations to user characteristics, we find that safety preferences significantly mitigate refusal penalties, whereas preferences for helpfulness, fluency, and creativity do not. Heavy users penalize refusals more strongly, an association substantially mediated by lower safety preferences. These findings suggest that acceptance of safety-aligned behavior depends primarily on users’ safety orientation rather than preferences for instrumental response qualities. Our results highlight the limitations of evaluating LLM safety alignment through aggregate user preferences and underscore the importance of accounting for user heterogeneity in evaluation benchmarks and deployment decisions.
No takes yet. Share an insight, caveat, or question.
Pasch et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: