Prompting Better Feedback: A Study of Custom GPT for Formative Assessment in Undergraduate Physics

This study explores the use of a custom generative AI (GenAI) tool, built using a prompt-engineered instance of ChatGPT, to provide formative feedback on first-year undergraduate physics lab reports. A preliminary survey of 110 students identified writing style as an area of low confidence and highlighted strong demand for more actionable, detailed feedback. Students expressed greater comfort with GenAI in formative contexts, particularly when used alongside human assessors. The tool was refined through iterative prompt engineering and supported by a curated knowledge base to ensure accuracy, clarity, and pedagogical alignment. A mixed-methods evaluation with 15 students found that the feedback was useful, actionable, and clearly written, with particular praise for the suggested improvements and rewritten exemplars. Some concerns were raised about occasional inaccuracies, but students valued the tool’s structure, consistency, speed, and potential for interactive follow-up. These findings demonstrate that, when carefully designed and moderated, GenAI can serve as a valuable, scalable support tool within the broader formative assessment cycle for long-form scientific writing. The tool’s flexibility, clarity, and responsiveness highlight its value as a supportive resource, especially as generative AI technologies continue to evolve in educational contexts.