AI-assisted software development produces systems that pass tests, satisfy stated requirements, andrun in production. This paper argues that something more fundamental is also produced: theprogressive hollowing of the epistemic structure that makes software systems safe to change,maintain, and evolve.We introduce two concepts. A Living System is not a software system but the software systemtogether with the people who engage with it intellectually and the organization whose collectivememory — structured as a transactive memory system across all roles — is inseparable from it. AHollow System is the state produced when construction and operations are structurally decoupledfrom knowledge generation: externally correct, epistemically empty, stable on the surface, andthinning beneath.Three independent structural defects drive this transition. Mediocrity Bias arises because LLMsare trained on publicly available code oriented toward generality; their generative mechanism pullsgenerated artifacts toward the statistical center of a corpus that does not contain theorganizationally specific logic from which business software derives its value. The BypassedUnderstanding arises because every stage of conventional development and operations issimultaneously a knowledge-generation process — one that AI substitution structurally bypassesby shifting the human role from author to reviewer, progressively thinning the tacit understandingdistributed across all roles. The Unknowable Knowledge arises because design intent andjudgment, under AI-assisted development, are present only in attenuated, fragmentary, andunverifiable form: prompts are ephemeral, LLM reasoning is non-reproducible, and thehermeneutic scaffold through which experienced engineers reconstruct human design intent doesnot apply to the distributional failure modes of probabilistic generation.The Hollow System is distinguished from prior debt frameworks — cognitive debt, intent debt,epistemic debt — by its equilibrium property: it produces no visible distress signals and containsno natural corrective mechanism. The knowledge deficit is not a debt that accumulates and can berepaid. It is an absence that cannot be restored: the understanding that was not formed throughauthorship and intellectual engagement cannot be recovered from the artifacts that authorshipwould have produced.This paper provides the theoretical foundation assumed by companion papers on prescriptiveguidance (Sophia 2026a, 2026b). Its purpose is to name the phenomenon before it becomesuniversal.
Franny Philos Sophia (Wed,) studied this question.