Feedback Literacy: Holistic Analysis of Secondary Educators' Views of LLM Explanations of Program Error Messages

The implications of using large language model (LLM) tools for learning to program at secondary school level are largely unknown, and yet there is pressure for teachers to engage with these. To start addressing this gap, we investigated: RQ1: What are secondary educators' views on the potential classroom use of LLM program error message explanations? RQ2: In what ways can a feedback literacy perspective support the analysis of educators' views of potential classroom use of LLM program error message explanations? The responses of eight expert secondary school educators were gathered during a semi-structured, activity-based interview and qualitatively analysed. Fifteen themes were derived from their commentary, of which ten corresponded to enhanced program error message (PEM) guidelines. Yet, all themes correlated to feedback literacy theory, providing a more holistic view. The analysis revealed that educators preferred LLM explanations to guide and develop understanding rather than tell, that students should be supported to make judgements and action LLM-generated feedback. Combining PEM guideline and feedback literacy findings, we suggest augmented IDEs should be designed with educators and students in mind, and teacher professional development (PD) is needed. Research is needed to compare our findings with a wider range of educators and investigate what feedback literacy means for resource design, PD, and classroom practice in secondary and undergraduate contexts.