Generative artificial intelligence (GAI) algorithms have the potential to reshape education. Though GAI may help democratize access to education, it also presents many challenges for educators. Already, the ease of use of GAI has made it easier for students to bypass learning gains by prompting GAI for answers to assignments. Here, we assessed how ChatGPT performed on take-home assignments in a doctoral-level molecular biology course designed to train students in experimental design. Using Bloom’s taxonomy as a framework, we hypothesized that ChatGPT would perform similarly to doctoral students on lower cognitive levels involving memorization and underperform at higher levels that rely on critical thinking. Students outperformed ChatGPT, but surprisingly, this result was driven by ChatGPT’s poor performance on “remember” and “apply” tasks, which was partially improved by simple prompt engineering. To build assessments more robust to GAI usage, we developed and tested new free-response and multiple-choice assessments. We found a striking deficit in ChatGPT’s ability to interpret scientific graphs and raw data in both short-answer and multiple-choice questions, even when using a version specifically designed for image interpretation. Based on our results, we propose several tips for designing out-of-class assessments that promote student learning in the era of GAI.
Kwong et al. (Thu,) studied this question.