Robust assessment of changes in cellular networks

Cellular network service providers often have to conduct small scale testing in the operational network before a change (e.g., a new feature) is fully rolled out across the entire network. This is referred to as the First Field Application (FFA). However, assessing the effectiveness of FFA changes is challenging because of overlapping external factors: seasonality (foliage, leaves budding), weather (rain, snow, hurricanes, storms), traffic pattern changes due to big events (e.g., games at stadiums, students returning to school after holidays), and network events such as outages or other maintenance activities in different regions. In this paper, we first highlight the technical challenges in assessing the service performance impact of changes in operational cellular networks. We then propose Litmus, a new approach based on a spatial dependency model for robust assessment of changes. We evaluate the effectiveness of Litmus using real-world data from operational cellular networks (GSM, UMTS and LTE). Our operational experiences demonstrate accurate inferences of the service performance impact of changes in the field.