Linking multimodal cohesion, gaze behaviour and learning outcome in educational videos

Multimodal design in educational videos is a complex task, requiring the effective integration of various semiotic modalities to promote understanding among recipients. While superficially meeting proposed multimedia design principles, educational videos vary significantly in effectiveness. To explore this variation, the authors conducted a 2 × 2 experimental study with 45 students, examining the impact of instructor presence and dynamic vs static slide designs on eye gaze behaviour and learning outcome. Instructor presence was found to attract about 25 percent of visual attention away from presentation slides irrespective of their static or dynamic design, but learning gain was highest for dynamic videos with a visible instructor. This suggests that gaze behaviour and learning outcome do not correlate with the simple presence or absence of design features and a finer-grained approach is needed. The authors consequently applied a framework for multimodal cohesion analysis to relate individual design elements to a video’s discourse structure more tightly, finding that quantitative differences in cohesion predict differences in gaze behaviour between dynamic and static slide presentations, but irrespective of instructor presence. Future research will therefore need to formulate corresponding finer-grained pre-post knowledge tests to further correlate between discourse structures, gaze behaviour and learning outcome.