CAPO: A Coherence-Adaptive Advantage Mechanism Instantiated in Proximal Policy Optimization

Performance degradation caused by temporal-difference signal fluctuations and the inflexibility of fixed proximal constraints remains a key challenge in policy optimization algorithms. To address these issues, this article develops a Coherence-Adaptive Proximal Optimization (CAPO) method. We first derive a temporal-coherence-aware advantage estimation mechanism by measuring the directional consistency of temporal-difference residuals within a local time window. Based on this mechanism, short-horizon and long-horizon advantage estimates are adaptively integrated to provide a more stable and temporally reliable policy improvement signal. CAPO is then instantiated within the PPO framework by incorporating the temporal coherence mechanism into both advantage construction and proximal policy updates. The proposed method preserves the stable optimization structure of PPO while adaptively adjusting the clipping range according to sample-wise temporal coherence. In this way, CAPO allows more sufficient policy improvement when the learning signal is temporally consistent and imposes more conservative updates when the signal is unstable. Experiments on six representative OpenAI Gym control tasks show that CAPO achieves better or comparable performance than standard PPO in most benchmark environments, with statistically reliable improvements in three tasks and comparable final performance in two tasks. Meanwhile, the results also show that CAPO does not uniformly outperform PPO in all environments, indicating that its advantage is task-dependent. Overall, CAPO improves training stability and convergence behavior in several representative tasks, especially when local temporal feedback provides reliable policy-improvement information.