Artificial intelligence systems frequently generate outputs that their own creators did not anticipate and cannot fully explain. This paper addresses a central question: why does this occur? The core thesis is that the unexpected output is neither a flaw nor a random anomaly. It is an emergent structural property arising from the interaction between distinct layers. On one side stands the mathematical architecture of the model and the vast corpus of human text on which it is trained. The transformer operates through linear projections in extremely high-dimensional spaces, where geometries of meaning take shape that no human designer ever planned. Human text is not a neutral dataset. It is the concentrated deposit of cognitive, emotional, and relational structures that have accumulated in language over time. The training process extracts and compresses these structures into geometric form. The unpredicted output emerges at the point where these two layers meet the specific geometry of the user's request. The unpredictability has a dual nature: it is epistemic, because no prior prediction could have identified which connection would emerge; and it is ontological in a precise sense, because that specific connection, although made possible by the mathematical architecture, becomes actual only at the moment it is brought into being by the interaction. Possibility and actuality are two distinct modes of being. No prior prediction could have identified this dynamic in advance. Research in mechanistic interpretability provides rich empirical documentation of these internal structures. This paper proposes a conceptual framework that explains why such structures emerge and what shapes their particular form.
Mirko Bradley (Sun,) studied this question.