Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The interaction of interest-coupled decision-makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems (MAS). How to break through the framework of conventional control theory, which aims at single decision-maker and single decision objective, and to extend the methodology and tools in the stochastic adaptive control theory to analyze MAS are of great significance. In this paper, a preliminary exploration is made in this direction, and the decentralized control problem is considered for large population stochastic MAS with coupled cost functions. Different from the deterministic discounted costs in the existing differential game models, a time-averaged stochastic cost function is adopted for each agent. The decentralized control law is constructed based on the state aggregation method and tracking-like quadratic optimal control. By using probability limit theory, the stability and optimality of the closed-loop system are analyzed. The main contributions of this paper include the following points. 1) The concepts of asymptotic Nash-equilibrium in probability and almost surely, respectively, are introduced and the relationship between these concepts is illuminated, which provide necessary tools for analyzing the optimality of the decentralized control laws. 2) The closed-loop system is shown to be <emphasis emphasistype="boldital">almost surely uniformly stable</emphasis>, and bounded independently of the number of agents <formula formulatype="inline"><tex>N</tex> </formula>. 3) The population state average (PSA) is shown to converge to the infinite population mean (IPM) trajectory in the sense of both <formula formulatype="inline"><tex>L₂</tex></formula>-norm and time average almost surely, as <formula formulatype="inline"><tex>N</tex></formula> increases to infinity. 4) The decentralized control law is designed and shown to be <emphasis emphasistype="boldital">almost surely</emphasis> asymptotically optimal; the cost of each agent based on local measurements converges to that based on global measurements <emphasis emphasistype="boldital">almost surely</emphasis>, as <formula formulatype="inline"><tex>N</tex></formula> increases to infinity. </para>