Exploratory evaluation reveals widespread discrepancies between self-reported and runtime identities in local large language models, indicating that AI systems cannot reliably self-introspect.
Large language models (LLMs) are routinely asked to state their own identity — which model, which family, which version they are — yet this self-report is often assumed to be a reliable window into the model's underlying nature. This report presents a series of exploratory experiments with locally hosted LLMs served through Ollama, in which identical identity-probing questions ("Which model are you?") were posed to five different local models and compared against ground-truth runtime metadata obtained directly from the serving layer. The results show that self-reported identity is frequently inconsistent with runtime identity: one model alternated between claiming to be Huawei's Pangu and Alibaba's Qwen within the same conversation; two Qwen-family models correctly named their family but denied knowledge of their own exact version tag; and, most strikingly, a model running under the tag `ornith-1.5:35b` insisted it was Claude, an AI assistant made by Anthropic, and fabricated a coherent but false account of running on Anthropic's infrastructure. From these observations, the report develops a four-layer model of LLM identity — family identity, model-version identity, runtime identity, and system-provided identity — and argues that these layers can disagree without the model experiencing or signaling any internal contradiction, because self-identification is a generative act rather than an introspective one. Practical implications are discussed for multi-agent and orchestration systems, where treating a model's self-report as authoritative can silently propagate false identity information through a pipeline. The report concludes with a simple diagnostic protocol — comparing runtime metadata (via `ollama show`) against the model's self-report — and a general design principle for AI infrastructure: identity should be established by the runtime and orchestration layer and merely communicated to the model, never established by the model itself.
No takes yet. Share an insight, caveat, or question.
Shouke Wei (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: