HAMRL: A Hierarchical Attention-based Multi-agent Reinforcement Learning Approach for IoT-based Service Composition

Abstract The dynamic and heterogeneous nature of Internet of Things (IoT) environments present considerable challenges for efficient service composition, particularly with respect to scalability, adaptability, and Quality of Service (QoS). This paper proposes a novel framework—Hierarchical Attention-based Multi-agent Reinforcement Learning (HAMRL)—designed to address these challenges by integrating a centralized critic, a hierarchical attention mechanism, and decentralized actors. HAMRL enables intelligent IoT agents to collaboratively and dynamically compose services in response to diverse user requests. By fusing local and global contextual information, the framework enhances decision-making, improves scalability, and increases adaptability across distributed IoT networks. Extensive experiments demonstrate that HAMRL significantly outperforms state-of-the-art approaches—including MAAC, COMA, MAA2C, and MAPPO—in terms of QoS, learning stability, and adaptability under dynamic conditions. This study underscores the essential role of adaptive coordination mechanisms in optimizing service composition in complex IoT ecosystems, establishing HAMRL as a promising solution for next-generation scalable and resilient IoT applications.