Evaluating the Use of the SCAI Chatbot to Support Student Engagement and Academic Preparation in Secondary-School Physics

Despite the widespread adoption of educational technologies, a critical gap remains in Generative Artificial Intelligence (GenAI) tools that are both pedagogically grounded and responsive to the practical needs of teachers and students. This study examines the use of the Scaffolding Cognitive Artificial Intelligence (SCAI) chatbot to support students’ pre-class preparation and classroom engagement in secondary-school physics. The SCAI chatbot was designed by applying scaffolding theory and aligning chatbot interactions with the educational curriculum, with the aim of providing adaptive explanations, formative questioning, and teacher-facing preparation analytics. A Design Science Research (DSR) approach with a mixed-methods design was employed. Quantitative data were collected through pre-class preparation tests, while system-generated log data captured student interaction patterns. Qualitative data were obtained through semi-structured interviews with both teachers and students. The findings suggest that SCAI-supported preparation was associated with higher student preparation and more active engagement when compared with textbook-based preparation. Interview and log data further indicated that the chatbot helped simplify difficult concepts, support students’ confidence, and provide teachers with useful information about students’ progress before classroom instruction. However, because the study was conducted in a limited educational context and the intervention combined AI-supported interaction, scaffolding, flipped preparation, and teacher analytics, the findings should be interpreted as exploratory evidence of a meaningful association rather than definitive proof of causation. Overall, the study contributes to understanding how a scaffolding-based GenAI chatbot can support pre-class preparation and teacher-informed instructional planning.