Effectively prioritizing tests in development environment

Software testing helps ensure not only that the software under development has been implemented correctly, but also that further development does not break it. If developers introduce new defects into the software, these should be detected as early and inexpensively as possible in the development cycle. To help optimize which tests are run at what points in the design cycle, we have built Echelon, a test prioritization system, which prioritizes the application's given set of tests, based on what changes have been made to the program.Echelon builds on the previous work on test prioritization and proposes a practical binary code based approach that scales well to large systems. Echelon utilizes a binary matching system that can accurately compute the differences at a basic block granularity between two versions of the program in binary form. Echelon utilizes a fast, simple and intuitive heuristic that works well in practice to compute what tests will cover the affected basic blocks in the program. Echelon orders the given tests to maximally cover the affected program so that defects are likely to be found quickly and inexpensively. Although the primary focus in Echelon is on program changes, other criteria can be added in computing the priorities.Echelon is part of a test effectiveness infrastructure that runs under the Windows environment. It is currently being integrated into the Microsoft software development process. Echelon has been tested on large Microsoft product binaries. The results show that Echelon is effective in ordering tests based on changes between two program versions.