How to Evaluate the Legal Defensibility of High-Stakes Tests

Abstract When tests are used for high-stakes decisions, there is a strong possibility that individuals for whom an unfavorable decision is made will bring a legal suit against the developer and/or user of the test. In this article, we address the general issue of how to determine whether a test has been developed in a legally defensible manner. We discuss general legal issues, specific cases that bear on different types of test use, and some evaluative dimensions and evidence of test quality. Existing case law is based on statutory and constitutional requirements. The 1964 and 1991 Civil Rights Acts prohibit discrimination in employment. Both disparate treatment and disparate impact are issues. Most case law is based on disparate impact, which does not require evidence of discriminatory intent. If there is a showing of disparate impact, a test can still be used if it is shown to be job related and professionally developed. The U.S. Constitution's 14th Amendment requires equal treatment and due process. In testing, these requirements include a legitimate relationship between a requirement and the purpose, sufficient advance notice of a testing requirement, and opportunities for fair hearingdappeals. An examination of court cases suggests that, if tests are constructed and used according to existing standards, they should withstand legal scrutiny. Builders and users of high-stakes tests must attend to contractual arrangements and oversight of contractors, validity evidence, reliability evidence, opportunity-to-learn evidence, avoidance-of-bias evidence, and setting legally defensible cut scores.