Data-driven asynchronous optimal control for 2-D MJRSs with structural constraints via reinforcement learning

This article investigates the data-driven asynchronous structured optimal control problem for discrete-time two-dimensional (2-D) Markov jump Roesser systems (MJRSs) with unknown dynamics. The structural coupling of 2-D systems motivates a structured optimal control formulation. Moreover, the proposed framework addresses the challenges associated with asynchronous mode information and unknown system dynamics. To resolve these issues, a reinforcement learning (RL) framework is established to solve the derived asynchronous discrete-time algebraic Riccati equation (ADARE) using only measurable input-state data. Two data-driven algorithms are developed to achieve this objective. First, an off-policy policy iteration (PI) algorithm is constructed to reconstruct the optimal controller. Second, a value iteration (VI) algorithm is proposed to circumvent the requirement of an initial stabilizing policy inherent in the PI algorithm. Crucially, the VI approach facilitates online learning from arbitrary initial conditions. Theoretical analyses guarantee the convergence of both algorithms to the structured optimal solution and ensure the asymptotic stability of the closed-loop system. Simulation results demonstrate the effectiveness of the proposed framework under asynchronous conditions.