Analysis and Autonomic Elasticity Control for Multi-Server Queues under Traffic Surges

Many computing environments consist of a multitude of servers that process requests that arrive from a population of customers. Incoming requests that find all servers busy have to wait until a server becomes idle. This type of queuing system is known as a G/G/c system and has been extensively studied in the queuing literature under steady state conditions. In this paper we study multi-server systems that are subject to workload surges during which time the average arrival rate of requests exceeds the system's capacity. This paper's main contributions are (1) The derivation of a set of equations to estimate the impact of workload surges on response time; (2) A simulator for a G/G/c system to evaluate the accuracy of the equations in (1); and (3) The design, implementation, and extensive evaluation of an autonomic controller for multi-server elasticity that uses the equations derived in (1). The results show that our equations estimate with great accuracy the impact of surges on response time and that our autonomic controller is able to successfully determine how to vary the number of servers to mitigate the impact of workload surges.