Model-Driven Elasticity Control for Multi-Server Queues under Traffic Surges in Cloud Environments

Many computer systems, such as Internet datacenters and cloud computing environments, consist of a multitude of servers that process user requests. Incoming requests that find all servers busy have to wait for an idle server. This type of queuing system, known as a G/G/c system, has been extensively studied in the queuing literature under steady state conditions. This paper proposes two autonomic controllers for multi-server systems subject to workload surges of generic trapezoidal shapes during which the average arrival rate of requests exceeds the system's capacity. The main contributions of this paper are: (1) The design of a horizontal and a vertical autonomic controller for server elasticity under trapezoidal shaped workload surges, and (2) The implementation and extensive evaluation of the horizontal controller using both synthetic arrival streams and traces from a Google cluster. The results show that our autonomic controller is able to successfully vary the number of servers to mitigate the impact of the trapezoidal shaped workload surges.