ChatGPT in the Classroom: Evaluating its Role in Fostering Critical Evaluation Skills

Abstract The increasing prevalence of artificial intelligence in educational domains raises both opportunities and challenges in the context of academic integrity and pedagogical efficacy. This study outlines an innovative project that investigates the use of ChatGPT as a tool for enhancing the critical evaluation skills of master’s students in biomedical science. Using a dual approach combining AI-generated essay writing with subsequent student-led critical evaluation, this project sought to foster deeper critical evaluation skills in learners. By having participants critically assess AI-generated essays, supported critical evaluation based on peer-reviewed literature, the project aimed to deepen their evaluative skills. Outputs from the tasks were compared against academic benchmarks considering factors such as marks, writing, and overall quality. Participant perceptions were collected through a combination of a focus group session and an evaluation questionnaire. The key finding of this project was that while ChatGPT demonstrated proficiency in structural coherence and grammatical accuracy, it did not augment academic performance– participant marks for the AI-generated essays aligned closely with their overall module marks, showing no overall improvement. However, this study did see an increase in marks for participants’ critical evaluations. This suggests that ChatGPT was more effective as an assessment tool when used for critical evaluation tasks, aligning with pedagogical emphasis on nurturing critical evaluation skills. User interaction with AI emerged as a significant variable that influenced the tool’s efficacy, highlighting the need for a nuanced approach to its integration into educational settings. The study concludes that while ChatGPT offers promising avenues for both drafting and assessment, and demonstrated a high level of factual accuracy, it is not a substitute for human-led academic enquiry, and students preferred writing their own essays.