Zero‐sum game for nonlinear multiagent systems with full‐state constraints

Abstract This paper addresses the zero‐sum game problem for strict‐feedback nonlinear multiagent systems with full‐state constraints. Specifically, this paper focuses on the zero‐sum game scenario, wherein multiple agents aim to optimize the control strategies while considering the conflicting objectives of their opponents. To handle the full‐state constraints, a one‐to‐one nonlinear mapping technique is employed to convert the original strict‐feedback system into a more manageable pure‐feedback system without state constraints. In order to find a Nash equilibrium for virtual control signals and external disturbances, a simplified reinforcement learning algorithm is proposed, which tackles the challenges posed by solving the Hamilton–Jacobi–Isaacs equation. Unlike the existing optimal control strategies that deal with matching conditions, the optimal control strategy for strict‐feedback nonlinear systems needs to address the computational complexity issue arising from the repeated derivation of the virtual controller. To overcome the high‐order virtual controller problem, an approach based on the dynamic surface technique is introduced. By incorporating an approximation term of the high‐order virtual controller into the value function, the computational complexity challenge is effectively resolved. Based on the Lyapunov stability theorem, it is proved that all signals of the closed‐loop systems are semi‐global uniformly ultimately bounded and the tracking control performance can be guaranteed. Finally, simulation results are given to verify the effectiveness of the proposed control strategy.