Assessing AI-Generated Feedback Quality: Instructor Perspectives on Fine-tuned GenAI for Higher Education Student Writing

This is the authors' abstract. We don't add key points for this paper.

This convergent mixed-methods study investigates the integration of Generative Artificial Intelligence (GenAI) for providing formative feedback in higher education.The study explores the perspectives of eleven experienced instructors regarding GenAI-generated feedback in an online graduate program.Participants reviewed sample feedback and shared their evaluations via a numerical questionnaire, an open-ended survey, and a focus group discussion.Results indicated that instructors generally viewed GenAI feedback positively, noting its relevance, clarity, actionability, usefulness, and comprehensiveness, highlighting its effective alignment with course rubrics and comprehensive guidance.However, challenges were also noted, including redundancy and overly lengthy suggestions that could overwhelm students.The study concludes with recommendations for refining GenAI to enhance its feedback effectiveness and improve the learning experience of higher education students. Characteristics of human and AI feedback on written workExisting analyses of university instructors' formative feedback on written work (e.g., Hyland & Hyland, 2001;Pearson, 2022) have revealed that such feedback typically includes praise, criticism, and suggestions aimed at motivating students, enhancing their self-efficacy, and providing actionable guidance for improvement.These comments are grounded in specific pedagogical goals and outcomes (Connors & Lunsford, 1993;Straub, 1997).Furthermore, instructors often mitigate criticism through strategies such as hedging, using question forms, and employing paired-act patterns (where criticism is softened by praise, often within the same sentence) (Hyland & Hyland, 2001).These techniques have been shown to prevent negative emotional responses that could hinder students' ability to revise their work effectively (Pearson, 2022).Additionally, studies have revealed recurring patterns in the organization of instructor feedback.For instance, Mirador's (2000) analysis of feedback on postgraduate students' writing in the UK identified a common rhetorical structure.Feedback often begins with general impressions and a recapitulation of ideas, followed by an articulation of strengths and weaknesses, suggestions for improvement, and an overall judgment of the work.This structure, referred to as the clinching pattern, ensures that feedback is comprehensive and goal-oriented, fostering both reflection and action (Holmeier et al., 2018).In recent years, AI-generated feedback has emerged as a complement to human feedback.