Experiment finds that LLM chatbots improve pedagogical education in informatization, highlighting both benefits and challenges.
Problem and aim. Modernization of higher education involves the active use of virtual assistants and dialog agents in the training of digital school mentors. However, the use of large language models (LLM), which are complex systems for generating text and conducting a dialogue with users, requires taking into account their advantages and limitations. The aim of the study is to identify the possibilities of using chatbots based on LLM to improve the quality of training teachers in the field of informatization of education. Research methods. LLM is a deep learning model that uses and trains massive data sets to excel at language processing tasks. Master's degree students use the resource LMSYS Chatbot Arena, a crowdsourcing open platform for LLM assessment. The experiment was conducted at Vyatka State University (the Russian Federation). It involved 58 master's degree students of the training program 44.04.01. Pedagogical education, training program specialization “Informatization of education”. The integration of LLM-based chatbots into the didactic process was carried out within the framework of the discipline "Information technology in pedagogical activities". According to the structure of the corresponding work program, the author's testing consisting of 5 sections was developed. In statistical data processing, the Pearson's chi-squared test was used. KEYWORDS Results. The article describes what master's degree students used LLM-based chatbots for: defining concepts, analyzing examples; studying the interface and capabilities of the platform; practicing skills and understanding the principles of formulating prompts using simple examples; using the service for scientific research; formulating methodological recommendations for a novice user. The peculiarity of information interaction is that participants independently (taking into account the specifics of training) define entities in the text, formulate prompts, and select models. Statistically significant differences in the qualitative changes that occurred in the pedagogical system were revealed (χ2 = 6.820, p < 0.05). Conclusion. The didactic possibilities of using chatbots based on LLM are defined: generation of personalized content, multilingualism, activation of cognitive activities, intensification of feedback. Difficulties are also highlighted: a wary attitude towards new technological solutions and the need for additional resources to understand their operating principles; ethical issues.
No takes yet. Share an insight, caveat, or question.
Isupova et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: