This study investigates how undergraduate and graduate translation students in post-secondary education engage with and evaluate Large Language Model (LLM)-generated feedback through a mixed-methods approach, analyzing acceptance rates, influencing factors, decision rationales, and perceived limitations. 78 students majoring in translation (55 undergraduates, 23 postgraduates) completed translation tasks spanning six text types and received ChatGPT-3.5-generated feedback. Participants made binary accept/reject decisions with immediate written rationales, followed by semi-structured interviews to explore evaluative criteria and perceived deficiencies. Quantitatively, participants accepted an average of 68.2% of LLM suggestions, demonstrating receptive yet selective engagement, with no student accepting or rejecting all suggestions. Acceptance was most strongly shaped by text type, with technical and news texts receiving the highest approval and literary and tourism texts the lowest. Baseline AI attitude and proficiency further moderated engagement, as optimists accepted more suggestions than skeptics, higher-proficiency students within each academic level demonstrated greater criticality than their lower-proficiency peers, and postgraduates overall exhibited more selective evaluation than undergraduates. Qualitatively, students accepted feedback that corrected objective errors, improved fluency, or resolved uncertainty, but rejected suggestions due to cultural or contextual misunderstandings, preservation of personal style, unconvincing justifications, or risk aversion in high-stakes texts. Rejection often triggered deeper engagement, including self-revision, external verification, and dialogue with the AI. Thematic analysis revealed key deficiencies in LLM feedback, such as cultural blind spots, stylistic flattening, and contextual myopia. These findings highlight that students’ engagement with LLM feedback is shaped by the interplay of task characteristics, individual dispositions, and domain expertise, underscoring the need for translator training programs to develop feedback systems that are contextually aware, stylistically adaptive, and dialogic in post-secondary translation education.
Chen et al. (Thu,) studied this question.