Methodological Rigor and Justice in Language Assessment: Correcting the Course A Response to Owen and Xu’s “Maintaining Methodological Rigor in Language Assessment: A Response to Randall et al.’s Justice-Oriented Antiracist Validity Framework”

In this Viewpoint, we respond to Owen and Xu’s critique of our justice-oriented antiracist validity (JAV) framework, first advanced in “Our validity looks like justice. Does yours?” (Randall et al.). We argue that their response rests on a false dichotomy: that we must either repair educational measurement incrementally or abandon it altogether. JAV, we clarify, is not a replacement for validity theory but a critical extension of it, building explicitly on Kane’s interpretation-use argument and Mislevy’s sociocognitive foundations. We address four of Owen and Xu’s critiques. First, we take issue with their critique of QuantCrit, which confuses methodology with epistemology. As we explain, QuantCrit was never proposed as a new set of statistical techniques; it is a critical orientation toward the production, interpretation, and use of quantitative evidence. Second, we concur with Owen and Xu that a more expansive view of the history of testing is warranted. Unintended consequences of testing are often the result of a test’s long-term use, which ends up advantaging those with more power. Cultural systems of evaluation are always tied to systems of power, and in tracing the influence of assessment systems, we must always place them in their historical context to understand who is being advantaged and disadvantaged by them. Third, we concur that within the brief space of a Viewpoint article, it is impossible to provide a sustained treatment of Rawls’s contributions. Rawls alone is insufficient; for that reason, in our research we place his work within a broader conversation that includes Iris Marion Young, Charles W. Mills, Martha Nussbaum, bell hooks, Kimberlé Crenshaw, and Patricia Hill Collins. Finally, JAV was developed from a North American use case. Acknowledging that North America is not the only standpoint, we have worked with other researchers to develop and apply new models of principled design and validation across a range of contexts within and beyond the United States and Canada. JAV is not the totalizing project described by Owen and Xu. It is about procedures and consequences, a provocation made in service to those who are impacted by the work we do.