With the increasing complexity of safety-critical systems, reinforcement learning (RL) agents are required to optimize multiple objectives while facing progressively higher safety requirements in dynamic and nonstationary environments. Existing multiobjective reinforcement learning (MORL) methods typically assume stationary dynamics and rely on fixed reward structures, limiting their ability to maintain safety-performance trade-offs under environmental shifts. To address this limitation, we propose MMOSRL, a hierarchical metacognition-inspired reinforcement learning framework that introduces a dedicated metacognitive control layer above the RL agent. The metacognitive layer is structured into three components—knowledge, monitoring, and reflection—which collectively enable safety feature decomposition, real-time safety evaluation, and adaptive reward regulation. In particular, safety degradation is detected through a normalized survival score, and a dynamic reward reshaping mechanism is triggered via Safe Bayesian Optimization to adjust reward hyperparameters within a constrained search space. This design allows the agent to continuously realign its learning objective with evolving safety requirements without retraining. Simulation experiments conducted in autonomous driving scenarios demonstrate that in the most challenging high-density traffic conditions, MMOSRL achieves a 3.9% absolute improvement in success rate over the state-of-the-art baseline (PSL-MORL), while reducing collision and lane departure rates by 2.8% and 1.1%, respectively. The results validate that incorporating structured metacognitive control effectively enhances robustness and safety compliance in multiobjective, nonstationary environments.
Yang et al. (Mon,) studied this question.