Decentralized Learning based Optimal Design for RIS-assisted Multi-user Ad-Hoc Network: A Multi-Player Multi-Armed Bandits Approach

This paper focuses on decentralized dynamic resource allocation optimization for ad-hoc network communication using Reconfigurable Intelligent Surfaces (RIS) and a reinforcement learning approach. Device-to-Device (D2D) communication and RIS are highlighted for improving spectrum efficiency in wireless networks. Current centralized D2D schemes incur high signaling overhead with global information requirements, while distributed schemes lack global optimization. The proposed framework employs an Outer-Loop and Inner-Loop strategy, leveraging a Multi-player Multi-armed Bandit method in the Outer-Loop and the Twin Delayed Deep Deterministic policy gradient algorithm (TD3) in the Inner-Loop. The convergence of outer and inner reinforcement learning achieves distributed optimal resource allocation over time, validated by numerical simulations demonstrating effectiveness.