A versatile control architecture based on reinforcement learning for diverse control tasks in robotic fish

Robotic fish typically work in complex and unpredictable marine environments while performing a variety of different control tasks. The development of a versatile control strategy that is self-adaptive to diverse control tasks in disturbance-filled environments remains a pressing necessity. Reinforcement learning (RL) holds significant potential for achieving a control mode akin to that of real fish. However, the versatility of a RL-based control architecture that integrates the characteristics of robotic fish continues to pose significant challenges. This paper utilizes reinforcement learning to develop a versatile control strategy architecture, designed to meet the control requirements for swimming directions of robotic fish under diverse control tasks. The deep deterministic policy gradient algorithm is employed to establish a systematic control architecture by integrating normalized state and action vectors, reward regulation, and neural networks, thereby forming a methodical control logic. Normalized framework improves its adaptability to non-training control tasks. Aiming at a wire-driven soft robotic fish, a movement simulation engine that incorporates compliant caudal and pectoral fins has been developed to facilitate control strategy training. Four specific control tasks, including trajectory tracking, dynamic object tracking, and multi-objective control, are experimentally evaluated. The versatility of the control architecture has been thoroughly demonstrated, along with its superior robustness and enhanced anti-disturbance capabilities. The proposed control architecture holds potential for broad application in robotic fish and other underwater robots across diverse control tasks.