A process‐oriented perspective on pre‐service teachers' self‐efficacy and their motivational messages: Using large language models to classify teachers' speech

BACKGROUND: Recent studies have examined the relation between teacher motivation, motivational messages and student learning but are limited to an achievement-related context, primarily using survey data. Moreover, our understanding of the relation between various teacher characteristics, such as teacher self-efficacy (TSE), and their motivational message use remains limited. AIMS: Our study tested whether teacher speech can be classified into self-determination (SDT)-based motivational messages and reliably assessed with a large language model (LLM). Additionally, we analysed the relation between pre-service TSE and their motivational message use. SAMPLE: = 34) who participated in a survey and were video-recorded while teaching. METHODS: First, we manually classified pre-service teachers' motivational messages based on transcripts and used human-rater annotations to fine-tune an LLM. Second, we analysed the relation between pre-service TSE and motivational message use. RESULTS AND CONCLUSIONS: The fine-tuned LLM demonstrated promising performance in assessing SDT-based motivational messages but needs further refining to assess thwarting messages. The analysis with human annotation showed that pre-service TSE for classroom management positively affected the frequency of relatedness-supportive messages. Pre-service TSE for student engagement increased the likelihood of never using a competence- or relatedness-thwarting message. Pre-service TSE for instructional strategies reduced the frequency of autonomy-supportive messages. LLM-based analyses showed slightly different results but did not contradict human annotation-based analyses.