Plannable Test Selection Criteria for FSMs Extracted From Operational Specifications

Model-based test generation (MBTG) is becoming an area of active research. Several MBTG approaches extract a finite state machine (FSM) from a given model, and use structural (mostly transition) coverage of the extracted FSM as a test selection criteria. In this paper, we demonstrate inadequacy of structural coverage criteria, and propose a set of test selection criteria for extracted FSMs. Our models are described in terms of operations provided by the system under test (SUT). Each operation is specified as a set of possible results each with a guard condition and a set of update actions on its parameters and the system state. The proposed test selection criteria are based on (1) mutations of guard conditions and update actions, (2) concept of a session, which targets errors of SUT not committing the updated system state to persistent storage, and (3) 2-way coverage of independent operations available in a given FSM state. We describe an AI planning based algorithm for finding a sequence of operation invocations to satisfy our proposed test selection criteria. We illustrate our test selection criteria, and report results of a case study which compares fault detection capability of our proposed test selection criteria with that of structural criteria.