Unit tests and component tests do make a difference on fault localisation effectiveness

Agile testers distinguish between unit tests and component tests as a way to automate the bulk of the developer tests. Research on fault localisation largely ignores this distinction, evaluating the effectiveness of these techniques irrespective of whether the fault is exposed by unit tests---where the search space to locate the fault is constrained to the unit under test---or by component tests---where the search space expands to all objects involved in the test. Based on a comparison of sixteen spectrum based fault localisation techniques, we show that there is indeed a big difference in performance when facing unit tests and component tests. Consequently, researchers should distinguish between easy and difficult to locate faults when evaluating new fault localisation techniques.