Multi-Criteria Deep Reinforcement Learning Energy-Aware and QoS-Driven Scheduling for Containerized Microservices in Docker Swarm Clusters

Efficient scheduling of containerized microservices is essential to ensure performance, scalability, and quality of service in cloud native environments. Docker Swarm comes with three container scheduling strategies by default: Spread, Binpack, and Random. These policies are based on static heuristics and lack awareness of dynamic resource conditions and application-level performance metrics. As a result, they often lead to resource imbalance and degraded performance under cumulative and chronic workloads. This paper proposes MCSCM-RL (Multicriteria Scheduling of Containerized Microservices-Reinforcement Learning), a scheduler evaluated on Docker Swarm clusters. The new scheduling approach integrates container-level metrics, such as CPU and memory utilization, and container counts, with application-level QoS metrics, such as average startup time and average response time. A three-node Docker Swarm cluster with one manager and two worker nodes are used to gather real-time data, which is then used to continuously train the best node selection policies using a Deep Q-Network. Extensive experimental evaluation is conducted under both clean and cumulative deployment scenarios for containerised microservices. The results show that the proposed scheduler consistently outperforms default Docker Swarm schedulers across all evaluated metrics. Against the reference container scheduling strategies, the proposed scheduler can attain a reduction of CPU utilization by up to 25%, improve memory utilization, have lower startup latency, and provide consistent response time behavior. The proposed scheduler avoids the instability associated with the Random scheduler while effectively mitigating resource peaks commonly observed with Spread and Binpack schedulers. The proposed scheduler offers an adaptive, quality of service-aware, and energy-efficient solution for scheduling of containerized microservices in Docker Swarm architecture. Future work will focus on applying predictive workload modelling, extending to larger heterogeneous clusters or Kubernetes.