Assessing a class of software tools

The roles and capabilities of LDRA software Testbeds and their appropriate environments have been described in a number of papers 1,2. The way in which management uses the tools as elements of a controlled software development environment is described in 3. The principal benefits of such use are that management has the assurance that software development standards are enforced, and has reliable information concerning project status. The explicit standards enforced by the use of these tools are described in detail in 2. One class of these standards is that of test effectiveness, which is measured primarily through three test effectiveness metrics reinforced by a code auditing capability.This paper attempts to quantify the benefits of using such a software Testbed in providing assurance of the absence of program errors. The attempt is made from two viewpoints, the theoretical and the experimental.The theoretical aspect is important because the practical use of a tool may fail to demonstrate that the tool can be a powerful detector of a class of errors simply because no errors of that type were present in the software sample validated.Finally the paper attempts to summarise some of the experiences gained through the use of the tools over a twelve year period.