PBO-SAC: Optimized Soft Actor-critic for a 7-DoF Pneumatic Humanoid Robotic Arm with Bowden Cable Transmission

Humanoid robots can be seamlessly integrated into human-robot interaction scenarios due to their human-like appearances. Pneumatic artificial muscles (PAM) are promising actuators for such robots due to their similarity to biological muscles, but their limited contraction ratio constrains both appearance and motion range of the robot. This work presents a 7-degrees-of-freedom (DoF) pneumatic humanoid robotic arm that mimics the human arm in both appearance and movement capabilities. A hybrid actuation scheme, combining direct PAM actuation at shoulder joints and PAM actuation with Bowden cable transmission at the distal joints, is adopted to enable anthropomorphic scaling with a lightweight and compliant structure. To address the control challenges posed by the nonlinear dynamics of PAMs and Bowden cables, Pneumatic Bowden cable Optimized Soft Actor-Critic (PBO-SAC), a model-free reinforcement learning framework, is proposed to enable efficient on-hardware control policy learning for the robotic arm. PBO-SAC incorporates posture-perturbed decoupled training and local recurrent fusion networks to ensure safe and smooth policy learning. Simulation results verify improvements in PBO-SAC, while hardware experiments on trajectory tracking and teleoperated stacking tasks further demonstrate the multi-DoF coordination control performance.