Supporting Fair Interpretation and Use of Test Results through Principled Score Report Design: A Case Study

Data dashboards, displays, and reports of aggregated student test score results are ubiquitous. However, education research and studies from the public health and data visualization fields highlight persistent issues with education data communication practices. Despite good intentions of using data to understand and inform solutions to student group score differences, the type of visuals often used in assessment reporting—bar charts and gap-trend line plots—can cause harm by unintentionally reinforcing misperceptions about student groups and thereby reinforcing lower expectations for student performance and reducing actionability. Through a co-design process with policy-level score users, education researchers, and a data visualization expert, we developed eight prototypes of group-level assessment scores using evidence-based design mechanics. The paper outlines the prototype development process, including iterative feedback loops with expert reviews and focus groups. From this exploratory study, we review the key lessons we learned in terms of textual and graphical features that hold promise for improving reporting by reducing attribution biases and providing more complete information to improve actionability.