Answers are grounded in this paper and cite their sources.
Field report demonstrates development failure modes in AI-assisted software engineering, highlighting the need for rigorous external verification to achieve deterministic execution.
This project began with a successful one-off result: ChatGPT could generate a good report from structured input. The challenge was turning that success into a deterministic software product that could do the same work repeatedly, offline, without depending on ChatGPT or any other online service at runtime. The application domain was natal astrology. It was chosen deliberately as an unusual engineering test case; the reasons for that choice, and why none of the engineering argument depends on accepting astrology, are discussed separately in the paper. The technical outcome was real software. The path to it was much less elegant. ChatGPT was often capable, willing, and astonishingly productive, but it also had a habit of quietly substituting descriptions for deliverables, building elaborate processes around unfinished work, and occasionally producing code or architecture that looked more formidable than it proved to be. The central lesson is not that AI cannot build serious software. It can. The lesson is that somebody still has to define “done” outside the model’s prose, keep attention on the actual deliverable, and make important claims survive observable execution. This account is for software practitioners. ChatGPT happens to be the tool in this story, but the larger patterns are unlikely to be unique to it: silently substituting a description for the requested deliverable, over-elaboration, weak verification, and process drift. The paper is a practitioner field report based on a multi-week AI-assisted software-development project. It presents the complete project story, including the development workflow, recurring failure modes, qualification and testing practices, clean-environment and packaging lessons, and the production-centered workflow that ultimately survived.
No takes yet. Share an insight, caveat, or question.
Leonid Nilva (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: