Hierarchical Adaptive PID Tuning for Agile Flight: A Safety-Constrained Reinforcement Learning Approach

Multirotor unmanned aerial vehicles (UAVs) suffer from significant control performance degradation during aggressive maneuvers, primarily due to aerodynamic nonlinearities and coupling effects. Conventional fixed-gain PID controllers struggle to simultaneously satisfy performance and robustness requirements across the wide flight envelope. To address this challenge, this paper presents a novel hierarchical safety-constrained reinforcement learning (RL) framework for adaptive PID tuning: the inner loop employs fixed gains, the outer loop leverages proximal policy optimization (PPO) for online adaptive gain scheduling, and linear matrix inequality (LMI) constraints delineate robust parameter boundaries for the adaptive exploration. Importantly, the LMI feasibility strictly guarantees theoretical stability for the fixed inner-loop parameters at the linearization vertices within a linear parameter-varying (LPV) framework. Concurrently, the online outer-loop RL stage is protected by safety boundaries and a Lagrangian penalty mechanism acting as an effective engineering safeguard rather than a rigorous global stability proof. Comprehensive high-fidelity simulation benchmarks demonstrate that, compared with a baseline fixed-gain PID controller, the proposed framework reduces overshoot by 18.5% in high-speed step responses and improves the overall mean RMSE by 15.0% across 100 randomized mixed-trajectory trials (with improvements of up to 40.9% in highly dynamic scenarios), yielding consistent gains in trajectory tracking accuracy and disturbance rejection despite uncertain model variations. By seamlessly blending control-theoretic hard constraints with RL-based soft-parameter tuning, the proposed architecture offers a safe and highly adaptive solution for large-envelope flight control, demonstrating strong engineering relevance.