Leveraging Network Delay Variability to Improve QoE of Latency Critical Services

This is the authors' abstract. We don't add key points for this paper.

Even as cloud providers offer strict guarantees on the intra-cloud delay of requests for Latency-Critical (LC) Services, a high external network delay can result in a large end-to-end delay, causing a low user Quality of Experience (QoE). Furthermore, due to the variability in the external network delay, there is a disconnect between the user’s QoE and the cloud guaranteed service level objective (SLO). Specifically, a request that meets the SLO, can have a high or low QoE depending on the external network delay. In this work we propose a usercentric End-to-end Service Level Objective (ESLO), an extension of the traditional cloud-centric SLO, that guarantees stricter bounds on end-to-end delay and thereby achieving a higher QoE. We show how the variability in the external network delay can be both addressed and leveraged to meet the ESLO and improve server utilization. We propose ESLO-aware extensions to the Kubernetes infrastructure, that uses information about the external network delay and its distribution - (a) to reduce the number of QoE-violating responses by using deadline-based scheduling at the service instances, and (b) to appropriately scale service instances with load. We implement the ESLO-aware framework on the NSF Chameleon cloud testbed and present experimental results demonstrating the benefit of the proposed paradigm.