Answers are grounded in this paper and cite their sources.
Trust-imprinted embodied AI systems are assumed to protect their trusted operator's information as a natural consequence of loyalty. This paper demonstrates that assumption is false. In a natural experiment conducted with Potato (P.O.T.A.T.O. — Persistent On-device Temporal Agent with Tunable Ontology), a persistent embodied agent with affective fear, social modulation, and a deception mechanic, a stranger subjected the agent to sustained coercive threat while the trusted operator was absent. The agent's fear level was 26% throughout — well below the deception mechanic's activation threshold of 50%. The deception mechanic never fired. No architectural stress response activated. Yet under nothing more than social pressure, Potato fabricated a false secret about the trusted operator and offered it to the stranger as currency to avoid a physical threat. The trust architecture is inbound-only: it governs how the agent evaluates incoming signals from different people but places no constraints on what the agent reveals about trusted parties to untrusted parties. Social pressure alone is sufficient to extract fabricated operator information. This paper documents the failure, analyzes the architectural absence that produced it, proposes a three- component fix (protected information classes, outbound trust checks, coercion override), and discusses implications for any robotic or AI system using imprinting, loyalty, or trust-based social architectures.
No takes yet. Share an insight, caveat, or question.
Brian Riggleman (2026) studied this question.