A Stability Theory of Chain-of-Thought Reasoning

Chain-of-Thought (CoT) prompting has emerged as a transformative technique for enhancing large language model (LLM) reasoning capabilities. However, a fundamental theoretical question remains unresolved: when does adding reasoning steps improve accuracy versus amplify error? We introduce a stability theory for multi-step neural reasoning that formalizes this dichotomy. We model reasoning as a time-homogeneous Markov process operating on a state space of intermediate thoughts and define the Reasoning Stability Coefficient (RSC), a theoretical quantity that determines whether longer reasoning chains contract toward truth or expand into hallucination. We prove three main results: (1) exponential stability, where uniformly stable reasoning chains provably improve accuracy; (2) error amplification under a two-state coarse-graining model, where persistent errors dominate the reasoning process; and (3) the existence of an optimal reasoning depth beyond which additional reasoning degrades performance ("overthinking"). We further connect our theoretical framework to practical LLM inference strategies, including early stopping, self-consistency, and tool augmentation. Our framework provides a principled theoretical foundation for understanding when reasoning agents should continue reasoning versus stop.