The advancement of Generative Artificial Intelligence (AI) chatbots, such as ChatGPT, presents significant and transformative challenges in higher education teaching and learning, such as assessment and evaluation practices. While this is acknowledged, there has been very little research into what this might look like in daily practice in higher education. This study explored these challenges in one area of higher education practice: developing students’ transferable skills, including writing, critical thinking, and information literacy among undergraduate engineering students at RMIT University, Melbourne, Australia. Using a cohort comparison design, this study evaluated the impact of ChatGPT on students' attainment of transferable skills. The effectiveness of AI tools in enhancing educational outcomes was assessed with a standardised assessment framework used by independent assessors to grade students’ reports. The results, analysed using the Mann-Whitney U test and the intraclass correlation coefficient, revealed significant improvements in critical thinking and information literacy among those students who used ChatGPT. The study also explored the ethical implications of using AI in educational settings and highlighted the need for rigorous academic standards and the implementation of measures to ensure the responsible use of AI technologies. While the preliminary findings suggest that AI tools, particularly ChatGPT in this study, can positively impact certain students’ skills, more detailed and controlled studies are necessary to validate these results and explore further the mechanisms through which AI tools influence learning and skill development.
Pang et al. (Tue,) studied this question.