This work uncovers structural decompositions and stability conditions in self-modifying systems, highlighting the embedding of reinforcement learning in Markov decision processes.
This work develops a fully mathematical framework for discrete-time self-modifying systems whose internal update laws and policies are themselves part of the state and evolve through the same Markov dynamics as the environment. The setting assumes only local observation: at each step the system has access solely to its current state, with no privileged meta-level, external clock, or separate optimisation oracle. Within this framework, the paper introduces structural decompositions of the state space into environment, evaluation, policy representation, and auxiliary coordinates, and defines an associated internal “law space”. A distinguished coordinate is then treated as an evaluation functional that acts as a Lyapunov function for the full self-modifying dynamics. Under purely local drift conditions on the one-step transition kernel, the paper proves uniform boundedness of the evaluation functional and almost sure attraction to low-evaluation regions, using standard Lyapunov theory for Markov chains lifted to the augmented state. The notion of an intelligent region is defined structurally, without reference to future trajectories: membership is determined by the existence of a compatible decomposition, a policy realisation map from internal representations to external policies, and a well-defined evaluation drift depending only on local coordinates. As a concrete realisation, the paper shows how value-based reinforcement learning algorithms on Markov decision processes embed into this law-space formalism via state augmentation, and how existing Lyapunov-based safe RL conditions can be lifted to the extended dynamics. The result is a set of natural law–type constraints under which long-run stability of internal evaluation is compatible with self-modification, expressed entirely in probabilistic and dynamical terms.
No takes yet. Share an insight, caveat, or question.
Takahashi K (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: