A Hierarchical Option-Critic Soft Actor-Critic Framework for Adaptive Intelligent Control of Multi-Degree-of-Freedom Motion Systems

Multi-degree-of-freedom (multi-DOF) motion systems are widely employed in precision manufacturing, intelligent robotics, semiconductor equipment, and vision-guided automation, where coordinated control across multiple axes is essential for accurate and stable operation. However, inter-axis coupling, nonlinear dynamics, actuator interactions, and external disturbances pose significant challenges to achieving precise control under dynamically varying conditions. Conventional model-based and data-driven methods often depend on predefined system models or static state-to-action mappings, which can limit their adaptability to complex and changing motion behaviors. To address these challenges, this study develops a hierarchical reinforcement learning framework, termed Option-Critic Soft Actor–Critic (OC-SAC), for adaptive continuous control of multi-DOF motion systems. The framework integrates the temporal abstraction of Option-Critic with the entropy-regularized continuous control of Soft Actor–Critic, enabling high-level option selection for long-horizon control strategies and low-level policy optimization for fine-grained motion adjustments. Double-Q learning and Prioritized Experience Replay (PER) are further incorporated to mitigate value overestimation and improve the utilization of informative experiences during off-policy learning. The proposed framework is evaluated using the Ant-v5 benchmark and a customized Gym-based multi-DOF error-state control environment. Experimental results show that OC-SAC provides improved convergence behavior and control performance compared with representative reinforcement learning algorithms, while maintaining effective error compensation in the customized multi-DOF environment. These results demonstrate the applicability of hierarchical reinforcement learning to adaptive control of complex multi-DOF motion systems under varying operating conditions.