Partition testing vs. random testing: the influence of uncertainty

This paper compares partition testing and random testing on the assumption that program failure rates are not known with certainty before testing and are, therefore, modeled by random variables. It is shown that under uncertainty, partition testing compares more favorably to random testing than suggested by prior investigations concerning the deterministic case: the restriction to failure rates that are known with certainty systematically favors random testing. In particular, we generalize a result by Weyuker and Jeng (1991) stating equal fault detection probabilities for partition testing and random testing in the case where the failure rates in the subdomains defined by the partition are equal. It turns out that for independent random failure rates with equal expectation, the case above is a boundary case (the worst case for partition testing), and the fault detection probability of partition testing can be up to k times higher than that of random testing, where k is the number of subdomains. Also in a related model for dependent failure rates, partition testing turns out to be consistently better than random testing. The dominance can also be verified for the expected (weighted) number of detected faults as an alternative comparison criterion.