Reinforcement Learning for Trajectory Tracking of a Ducted-Fan Air-Ground Amphibious UAV

Rotorcraft aerial robots offer unique advantages in aerial mobility, making them ideal for a wide range of missions. However, the high energy consumption inherent to flight significantly limits their practical deployment. In recent years, air-ground amphibious UAVs have emerged as a promising solution by integrating ground locomotion capabilities, enabling substantial energy savings and extended endurance while preserving aerial agility and maneuverability. Nevertheless, these multimodal systems pose new control challenges, such as mode transitions between air and ground, dynamic uncertainties during frequent takeoffs and landings, and the complexities of ground contact dynamics. To address these issues in a unified and scalable manner, reinforcement learning offers a compelling alternative to traditional hand-crafted control schemes. This paper focuses on an amphibious UAV with a ducted-fan configuration and develops a high-fidelity simulation environment with contact dynamics. A deep reinforcement learning framework is proposed to achieve robust trajectory tracking across diverse motion states, including aerial flight, ground cruising, takeoff, and landing. Compared with conventional feedback controllers, the trained policy demonstrates superior tracking accuracy. Furthermore, a novel potential field-based reward function—combining attractive terms at both long and short ranges—is introduced to enhance training efficiency and performance. Experimental results show consistent improvements over baseline reward designs across all motion modes, highlighting the framework’s effectiveness and generality.