ABSTRACT Interactive computer technology is deeply integrated into traditional teaching methods. The traditional teaching of physics experiments in secondary schools suffers from the inability of teachers to provide timely guidance to students, the difficulty of controlling experimental variables, and the lack of uniformity in evaluation criteria. To address the aforementioned issues, we have developed an innovative system to improve secondary school physics education using computer vision‐based interaction with virtual humans and sensors. The proposed system captures experimental data in real time so that student performance can be accurately monitored and assessed. Teachers can effortlessly configure experiments through simple coding, while the system leverages a multimodal macrolanguage model to offer contextual feedback and guidance. The system generates a virtual teacher that offers step‐by‐step guidance and real‐time feedback. Usability tests indicate that the system significantly improves student engagement and comprehension of complex physics concepts, highlighting its potential to transform traditional science education. The advantage of real‐time assessment and guidance in secondary school physics experiments is that it enables students to grasp abstract concepts in a more intuitive and comprehensible manner.
Wang et al. (Thu,) studied this question.