Using operational distributions to judge testing progress

Although most software test data adequacy criteria proposed as a way to assess the progress of testing rely on the coverage of the code, the coverage of the specification, or the percentage of the input domain that has been exercised, none of these are really good indicators of how thoroughly the software has been tested. Instead we propose that the assessment be based on the percentage of the probability mass associated with the test suite. This requires that data be collected to determine how the software is used in practice over a period of time, so that this can be properly assessed. In that way a project is able to accurately determine how testing is progressing.