Localized Multi-Agent Reinforcement Learning for Cooperative Management of Supply Chains

While reinforcement learning has significant applications in smart manufacturing, effectively coordinating multiple agents with limited communication remains a significant challenge. In this paper, we propose a localized multi-agent reinforcement learning approach specifically designed for serial supply chains. We formulate the supply chain management problem as a Markov decision process and uncover the exponential decay property of the [Formula: see text]-functions, which allows each agent to approximate the [Formula: see text]-functions using local observations and communications. Then, we propose the scalable natural actor–critic (SNAC) algorithm to solve the problem. The SNAC algorithm leverages localized coordination and reduces reliance on global information, thus addressing the challenges of large-scale and dynamic supply chain environments. Additionally, we conduct numerical experiments to demonstrate the effectiveness of SNAC in managing serial supply chains.