When Voice Meets Touch: Conflict Analysis in Mobile Applications

The recent advancement of the automatic speech recognition (ASR) contributes to the voice user interface (VUI), which is broadly embedded into mobile apps. The VUI implemented on modern mobile operating systems like Android naturally involves multiple threads, and brings new race issues and challenges in defining and identifying them. Specifically, when the GUI and VUI (GV) actions both access to the same resource simultanously, the data race named GV-race may occur. GV-race can lead to wrong behavior and even crashes. However, to the best of our knowledge, this problem has not been adequately studied. In this paper, we present the first study of GV-race on Android apps. However, the involvement of the VUI complicates the concurrency model, affects the temporal relationship and brings state space explosion in global analysis. To tackle these challenges, we firstly define primitives and their happen-before rules to abstract GV interaction patterns. Using these primitives, we are able to characterize and formally define GV-race. We then develop Roma (GV-race detector on mobile apps) to detect both app-level and system-level GV-race automatically. Through static program analysis, Roma extracts GV related call graphs for each pair of conflicting GV actions to reduce the state space, and generates a universal GV interaction graph using our pre-defined primitives. It encodes happen-before constraints to formally specify the freeness of GV-race, so that the detection of GV-race can be reduced to constraint solving with SMT solvers. We apply Roma to analyze 266 apps. Roma finds 52 apps with app-level GV-race and 56 apps with system-level GV-race. We confirm that 101 apps are true positives.