The goal of the present large-scale experiment was to test if a 90-minute, open access set of online learning modules with embedded practice and feedback would enhance students’ generative artificial intelligence (genAI) competency. We also examined whether impacts were experienced equitably by all included student demographic groups. Participants were 1,368 undergraduate and graduate students enrolled in 65 course sections. Sections were randomly assigned to treatment (engagement with the learning modules between pre- and post-tests) or control (no learning modules between assessments). Generative AI competency was defined as knowledge, skills, and attitudes and was measured at pre and post. The learning modules significantly improved treatment students’ genAI knowledge, prompt engineering skills, fact- and source-checking skills, and self-efficacy. There was a null effect of the modules on the skill of critically evaluating potential bias in genAI’s output. Improvements in genAI competency were equitable across all student groups (birth at sex, race/ethnicity, student level, first-generation status, and academic discipline). Such readily-available competency trainings are important for supporting the emerging necessity of knowing when and how to work with genAI in one’s education and future career. Limitations, implications, and future directions are discussed. • Created online learning modules to improve students’ genAI competency. • Ran a large experimental study testing the learning modules’ effectiveness. • Measured knowledge, attitudes, and ability to prompt-engineer and interpret output. • Our online learning intervention enhanced students’ genAI competency. • The treatment group experienced equitable learning outcomes across student groups.
Pensky et al. (Wed,) studied this question.