Background: Qualitative data is a rich source of information about the student experience. However, the amount of work necessary for effective analyses can make it less accessible for directing broad educational decisions. A wide array of software has been developed to provide quick summaries of qualitative data, but has yet to provide the in-depth analysis afforded by trained coders and educational experts employing established methodology such as thematic analyses. Recent research has suggested that current Generative Artificial intelligence (GenAI) offers a potential of quick and effective qualitative analyses comparable to human researcher. Aims: The present study partially replicated recent step-wise thematic analysis procedures (Naeem et al., 2023; Nyaaba et al., 2025), employing two separate GenAI (Chat GPT 4.5, Gemini 2.5). This partial replication aimed to test GenAI's ability to analyse students' short answer feedback regarding the use of a recently developed educational platform. Methods: 580 students (grades three to six) from one primary school in Japan participated in this study. The present study followed procedures developed (Naeem et al., 2023) and validated by recent research comparing GenAI and human coders (Nyaaba et al., 2025). Consistent with this research, students' open-ended responses to a request for feedback regarding learning experiences with a bespoke digital learning-to-read were analysed employing two GenAI. Results: Comparison of coding from each GenAI suggested robust overlap, but the potential for strong analysis through the integration of codes (15 in total). A random subset (20 feedbacks) was coded by each GenAI, coder inter-rated reliability was tested and suggested sufficient reliability (Cohen’s Kappa .61). Review of the final coding by ChatGPT4.5, suggested coherency and consistency. Results from ChatGPT4.5 for each of the remaining thematic analysis steps (themes, conceptualisation, conceptual model) were reviewed by the researchers. The final conceptual models describing the student experiences were used to construct directions for further refinement of the digital learning-to-read platform. The partial replication of Nyaaba et al.'s GenAI thematic analysis suggested that with careful supervision, GenAI are a powerful tool for efficient and substantive thematic analysis. Given the enormous amount of feedback collected through a range of different forums within education, using GenAI to make sense of this information an invaluable is opportunity for many educators and researchers.
Nakao et al. (2025) studied this question.