How to communicate evidence to teachers: Comparing the effects of verbal and visual effect size representation [Registered Report – stage II]

Background Teachers are encouraged to consider relevant evidence from educational research to improve the quality of their instruction. This approach requires them to have a reasonably accurate understanding of the effectiveness of educational interventions. However, currently prevalent ways of communicating effect sizes often lead to imprecise effect magnitude estimates. Aims We investigated how teachers estimate and perceive verbally and visually communicated effect sizes. Sample We recruited N = 120 English-speaking teachers ( M age = 43.5 years, 69% female) from the UK and USA via Prolific. Methods We employed a randomized 4 × 6 × 2 between-within-within-person experiment, with the outcomes of interest being teachers’ effect magnitude estimates, relevance, perceived informativeness, and perceived difficulty. Results Across conditions, teachers' estimated effect magnitudes correlated strongly ( τ = 0.70) with presented effect sizes, indicating that they can differentiate effectively between small and large effect sizes. Split by condition, teachers were less accurate in the verbal than in the visual condition (Cohen's d = − 0.31, 95% CI [ − 0.43 , − 0.19]). Additionally, teachers' person-standardized relevance perceptions correlated moderately ( τ = 0.35) with presented effect sizes. Moreover, they perceived verbally communicated Cohen's U 3 expressions as more informative ( r rb = − 0.34, 95% CI [ − 0.39 , − 0.29]) and less difficult ( r rb = 0.39, 95% CI [0.33 , 0.44]) to understand than visually communicated effect sizes. Conclusion Communicating effect sizes to teachers works verbally and visually. However, there appears to be a trade-off between the two presentation modes: higher accuracy in the visual versus greater user-friendliness in the verbal mode.