Abstract Students frequently exhibit only a partial understanding of the code they produce. Such gaps in comprehension may not become apparent until later stages of their studies, at which point misconceptions are more difficult to address and correct. Developing a deep understanding of code is particularly critical in the current context, where learners increasingly have access to generative artificial intelligence (GenAI) tools, such as GitHub Copilot. A commonly employed strategy to assess and promote code comprehension involves posing targeted questions on student submissions, enabling instructors to evaluate understanding directly. This method can also incidentally assist in identifying potential cases of plagiarism. However, while effective, this approach is resource-intensive and presents significant challenges in terms of scalability and sustainability. In response to these limitations, this work proposes an automated solution that leverages GenAI to generate multiple-choice code comprehension questions. We present an empirical study of this approach conducted within an introductory programming course, integrating the method within the CodeRunner automated assessment platform.
Goodfellow et al. (Wed,) studied this question.