Answers are grounded in this paper and cite their sources.
This theoretical account elucidates agency through mutual surprisal in various systems, suggesting implications for AI alignment.
What is an agent? This paper offers a candidate structural account: agents are systems sustaining mutual surprisal across a self-closing causal loop on its own closure timescale. The account is enumerated rather than universal — RNA, bacteria, Aplysia, and humans are in; thermostats, tornados, and most current LLMs are out — and extended beyond the enumerated cases by a named conjecture (Conjecture A). Universality is not claimed; universality without a state space cannot be cashed. The selection is theory-laden: the four in-cases share other properties (metabolic closure, autopoietic organization, thermodynamic openness) any of which a different framework could elevate. The case for mutual surprisal rests on what follows from it, not on the cases forcing it. What follows is a cross-framework pattern. Reinforcement learning, the free energy principle, predictive coding, active inference, and control theory share a minimization-shape objective whose bare optimum coincides with collapse of mutual surprisal across the loop; their framework-specific machinery performs structurally similar work in preventing the bare optimum. Reward hacking, mode collapse, hallucination, and dark-room dynamics are proposed as slices of this single structural pattern — a reading independently circled by the Proxy Compression Hypothesis (Wang et al., 2026) and the mesa-optimization framework (Hubinger et al., 2019). The operationalization: τ_c is identified with the temporal scale at which dynamical dependence (Barnett and Seth, 2023) across the agent-environment partition is minimized. The identification inherits validated measurement methods from the causal-emergence tradition (Hoel and collaborators). Two named conjectures concentrate the speculative content. Conjecture A (extension): the structural object extends to other systems satisfying the structural conditions. Conjecture B (content-non-selectivity): gap-monitoring oriented at proxy-requirement decoupling cannot be selectively oriented across content domains. A and B carry explicit falsification conditions in the body and are independent. If A holds in the AI direction, a capability-refusal correlation in deployed AI is predicted; if B additionally holds, the correlation is architectural. The AI extension is the most accessible test site for A. The framework is loop-prior and information-first, in the Walker-Davies-Pattee tradition; the contribution is synthetic.
No takes yet. Share an insight, caveat, or question.
Tamás Árpád Bartha (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: