Data-Driven Surrogate Pretraining and Policy Transfer for Rotary Inverted Pendulum Control: A Gazebo-Based Study

Learning control policies for underactuated systems can require extensive interaction with a physics simulator. This paper presents a unified surrogate-to-simulator policy-initialization framework for rotary inverted pendulum control. A mixed dataset of policy-driven and structured excitation signals, comprising more than 2.5 million transitions from 3000 Gazebo episodes, was used to train an action-conditioned long short-term memory (LSTM) dynamics surrogate. The learned four-state predictor achieved R2 values of 0.98, 0.96, 0.95, and 0.97 for motor angle, motor angular velocity, pendulum angle, and pendulum angular velocity, respectively, under the reported evaluation protocol. The surrogate was embedded as an interactive Soft Actor–Critic (SAC) environment for actor and value-estimation pretraining, followed by network transfer and fine-tuning in Gazebo. A quadratic regulation reward with a 10-point step reference yields a theoretical episodic upper bound of 10,000 over 1000 steps. The recorded Gazebo learning histories show earlier high-return episodes after surrogate pretraining than during training from scratch. The central contribution is an integrated controller-development method that links sequence-based system identification, learned-environment policy optimization, and target-simulator adaptation.