FIGARO: Hierarchical Reinforcement Learning for Scalable Microservice Management in the Computing Continuum

Nowadays, microservice-based applications are often deployed across a computing continuum that spans from edge devices (offering low-latency access but limited resources) to powerful cloud data centers. Balancing Quality of Service (QoS) constraints with cost efficiency in such a dynamic, heterogeneous environment remains a major challenge, especially when workloads fluctuate unpredictably. We introduce FIGARO (reinForcement learnInG mAnagement acRoss the computing cOntinuum), a hierarchical Reinforcement Learning (RL) framework for the dynamic resource management in the computing continuum. By observing component demands, resource characteristics, and QoS constraints, FIGARO guides layer-specific RL agents to autonomously adjust the number of active resources and handle unpredictable variations in the system state. Our approach includes an initial offline training phase, where agents gain insights from a heuristic-based expert, followed by an online deployment that adapts to evolving environment conditions. The experimental validation shows that FIGARO maintains QoS violations below 7% even when the distributions of components service times deviates significantly from the one considered during training, while reducing energy costs related to the resource usage by around 5% compared to baseline agents. The framework can therefore address large-scale, distributed computing environments and complex workflows, providing an effective solution to runtime microservice management in the computing continuum.