Improving Testing by Mimicking User Behavior

In-house tests are hardly representative of the rich variety of software behaviors exercised by real users in the field. To bridge the gap between in-house tests and field executions, we need ways to (1) identify behavior exercised in the field but not in-house, and (2) generate new tests that exercise such (or at least similar) behavior. In this context, we propose Replica, a technique that uses field execution data to guide test generation. Replica instruments the software before deploying it, so that field data collection is triggered when a user exercises an untested behavior. Then, when it receives the collected field data, Replica uses guided symbolic execution to generate executions that exercise this previously untested behavior. Our empirical evaluation shows that Replica can successfully generate tests that mimic field executions in terms of both behaviors exercised and faults detected. Our results also show that Replica can outperform a state-of-the-art input-generation technique that does not leverage field data.