A Control Method for Wheeled Bipedal Robots Using Improved PPO and Multidimensional Curriculum Learning

This is the authors' abstract. We don't add key points for this paper.

Wheeled bipedal robots offer enhanced agility, but as an underactuated system based on a wheeled inverted pendulum model, they exhibit strong nonlinearity and coupling, making traditional control methods ineffective on complex, unstructured terrain. This work proposes Adaptive Curriculum Proximal Optimization (ACPO)-a deep reinforcement learning approach for locomotion control that combines an improved PPO algorithm with multidimensional curriculum learning (MCL). First, by incorporating an adaptive clipping coefficient based on KL divergence (Importance-Weighted Clipping, IWC) and a PI-KL self-adaptive learning rate (SALR) into PPO, ACPO achieves efficient exploration in early training and stable convergence in later stages. Second, we propose an MCL framework that dynamically increases and fine-tunes reward weights, action-command complexity, observation noise, terrain difficulty, and external disturbances. Finally, large-scale parallel training and comparative experiments are conducted in the NVIDIA Isaac Gym simulation environment. Ablation studies demonstrate that, compared to baseline PPO, ACPO signif-icantly outperforms in terms of cumulative reward, episode length, orientation stability, fall frequency, velocity-tracking error, and mean joint torque-exhibiting superior robustness, energy efficiency, and cross-terrain adaptability.