Does Increased Choice over Learning Topic Improve the Effectiveness of Automated Feedback for Educators?

Educator agency in the form of choice over learning experiences is widely thought to enhance educator engagement and instructional improvement, yet causal evidence is scarce. We conducted a preregistered randomized controlled trial in an online computer science course with volunteer instructors who teach students worldwide. All instructors (N = 583) received automated feedback on their instruction, with half randomly assigned to have choice over the feedback topic. Choice alone did not increase feedback engagement or yield observable changes in practice, but it raised student attendance—an effect that was strongest for instructors who voluntarily engaged with additional training resources, including training modules and teaching simulations. For this subset of instructors, having choice over feedback had significant positive impacts on their instruction and student outcomes compared to the control group. This suggests that agency in choosing feedback topics may be most effective when combined with instructors’ intrinsic motivation to pursue self-directed improvement. Our study also demonstrates a scalable method for testing design principles in educator training and underscores the need to examine when, how and for whom agency might drive improvement.