Answers are grounded in this paper and cite their sources.
Distributed policy iteration (DPI) and distributed value iteration (DVI) are the two main algorithms of distributed reinforcement learning (DRL), which are used to solve distributed cooperative game (DCG) problems derived from distributed cooperative consensus control. The convergence speed of DPI is fast, but the initial control strategy must be admissible. The DVI algorithm avoids the shortcomings of the DPI algorithm, but its convergence speed is slow. To find a method to balance the DPI algorithm and DVI algorithm to solve the DCG problem in discrete-time linear (DTL) multiagent systems (MASs), a model-based multistep DVI RL algorithm is proposed in this article. First, the distributed leader-follower tracking control in general DTL MASs is transformed into a DCG problem. The Nash equilibrium solution of the DCG is obtained by using the proposed algorithm to solve the Bellman equation. Then, based on the analysis of the DVI and DPI algorithms, a multistep DVI algorithm with the advantages of these two iterative algorithms is proposed. Second, we demonstrate the convergence of the proposed algorithm and design a universal stability criterion to ensure that the obtained control strategy is admissible. Finally, the feasibility and effectiveness of the proposed algorithm were verified through numerical simulation experiments.
No takes yet. Share an insight, caveat, or question.
Qi et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: