Reinforcement learning at the edge of chaos

Reinforcement learning offers a powerful data-driven framework for solving complex control tasks. However, despite strong empirical performance in simulated domains, reinforcement learning methods remain rarely deployed in real-world systems. We argue that a key reason for this is because the learned control policies do not account for the predictability of the closed-loop control interaction. Consequently, these policies frequently induce chaotic state dynamics, where small changes to the control system can lead to dramatic and unpredictable long-term behaviour. This unpredictability poses a critical barrier for realworld deployment, as chaotic behaviour can lead to potentially unsafe outcomes. To address this issue, we propose two methods for embedding predictability into the training process. First, we introduce a Lyapunov-based regularisation method that constrains the divergence of predicted trajectories by penalising the predicted Maximal Lyapunov Exponent during policy updates. This regularisation encourages the controller to generate smoother, more predictable closed-loop dynamics and introduces a tuneable trade-off between task performance and predictability. Second, we develop a fine-tuning framework that updates pre-trained policies with a stabilising intrinsic reward function. This approach improves global stability without retraining from scratch and is compatible with any existing reinforcement learning controllers. Together, these contributions tackle a key limitation in current reinforcement learning practice: without guaranteed predictability, even high-performing policies remain unsuitable for real-world deployment. By explicitly shaping the dynamics of closed-loop behaviour, this work advances deep reinforcement learning towards robust and reliable control in real-world control systems.