Adaptive Auto-Scaling of Delay-Sensitive Serverless Services with Reinforcement Learning

Serverless services such as image recognition and natural language processing have strict response-time constraints. The incoming workloads and resource requirements of a newly deployed serverless service are always unpredictable due to the lack of available historical tracing data. Therefore, making effective auto-scaling decisions for these services is challenging. Open source serverless platforms often work in a best-effort manner, which cannot guarantee the response delay. Moreover, existing studies usually adopt threshold-based methods by configuring additional resource, which cannot well balance the trade-off between the quality of service and resource efficiency. To address the above issues, we propose an adaptive auto-scaling approach for delay-sensitive serverless services with reinforcement learning. First, we characterize the service's resource profile by exploring the performance improvement of different resource allocations with the reinforcement learning method. Then, we propose an adaptive auto-scaling method combining both horizontal and vertical scaling strategies based on the characterized profile to dynamically adjust the resource allocation. Finally, we select three typical services to validate our approach by comparing with two existing state-of-the-art auto-scaling methods. The experimental results show that our approach can accurately characterize services' resource profile, and effectively ensure the response delay constraints while achieving about 10.50% reduction of cost on average.