State-of-Practice in GUI-based System and Acceptance Testing: An Industrial Multiple-Case Study

Software testing is an essential means of evaluating software quality. System and acceptance tests aim to validate a system's conformance to its requirements on a high level of system abstraction. Therefore, they are generally performed by executing end-user scenarios through the system's graphical user interface (GUI). However, to the authors' best knowledge, there are no empirical studies that evaluate how GUI-based system and acceptance testing is performed in industrial practice. In this paper, we present a multiple-case study with the goal to investigate the state-of-practice of GUI-based system and acceptance testing at six software development companies of varying context. The main findings are that manual, GUI-based system testing is widespread and that automated GUI-based system and acceptance testing exists only on a small scale. Additionally, the study identifies core problems with GUI-based system and acceptance testing such as test tool limitations, high test costs and customer involvement in testing.