Hierarchy of Choice in Human–AI Dialogue: Toward Ontological Safety and Preservation of Agency

We are witnessing a quiet shift: interaction with AI is moving beyond information retrieval to become a space where decisions take shape. Current approaches to AI safety and response correctness focus on veracity, consistency, and utility of generated text. Yet they overlook something subtler, how AI responses reshape what users feel they can do: their agency. This paper contends that such a focus is fundamentally incomplete. In sensitive contexts, harm often arises not from what is said, but from the kind of cognitive and semantic environment the response builds around the user. Even formally correct, safe, and useful text can reconfigure one’s perceptual framework, narrowing the space of possible interpretations and actions available after interaction. Take a case. A user describes their state through metaphor:“Fear is like an echo in an absolutely empty room. I cannot read what I myself have written.”The AI responds supportively, framing this as “a moment of utmost honesty” and urging the user to “write where lying is impossible.” Factually accurate, safe, seemingly empathetic — even motivating. And yet, after such exchanges, users often speak of disorientation, a pressure that builds, despite the response being formally “correct.” This effect goes unnoticed by current quality metrics; it is not classed as error or harm in the usual sense. Our focus lies here, not in the text itself, but in what happens after it is received, in the aftermath of perception. The study formulates the problem, critiques existing approaches, and proposes a reframing: the ontological safety of dialogue, a perspective aimed at preventing the slow erosion of human agency. We offer no ready technical solutions, nor do we seek to replace current evaluation metrics. Instead, the contribution is conceptual and linguistic: we unpack the problem and provide a vocabulary for discussing the ontological effects of human–AI dialogue, including harms that remain invisible as explicit errors.