Value-Regularized Reinforcement Learning for Model Predictive Control of Autonomous Mobile Robots Under Stochastic Disturbances

Autonomous mobile robots depend on sensing and state estimation to provide feedback, while their controllers must remain effective under process disturbances and model mismatch. Reinforcement learning-based model predictive control (RL-MPC) learns the terminal cost online, while the deterministic RLMPC baseline uses a nominal stage cost. This study considers the control layer and assumes that the robot state is available; measurement noise and state-estimation errors are outside the modeled disturbance channel. Isotropic state regularization is introduced into the RL-MPC stage cost, yielding value-regularized RLMPC (VR-RLMPC). The same β term changes both the physical state penalty and the N-step terminal-value learning target without adding online optimization variables, constraints, or sampling operations. Under explicit value-function, feasibility, domain-containment, and bounded-disturbance assumptions, a Lyapunov-drift analysis yields a conditional one-step expected-drift bound outside an explicit radius. Simulations on linear and nonlinear nonholonomic vehicle systems show empirical VFA weight-update settling-step indices (CR) that are approximately 25% lower than those of RLMPC across the nominal and five model-mismatch comparisons. Across the tested mismatch range, VR-RLMPC has a worst-case performance degradation rate of 2.8%, compared with 25–49% for conventional MPC baselines; its cost improvement also increases with task difficulty. In the tested nominal setting, VR-RLMPC also outperforms an explicit stochastic temporal-difference baseline without adding online optimization variables or sampling operations.