Using ChatGPT is easy, using it effectively is tough? A mixed methods study on K-12 students’ perceptions, interaction patterns, and support for learning with generative AI chatbots

Abstract Generative AI (genAI) chatbots can be a powerful tool for learning, especially self-regulated learning, but they can also add more complexity to the learning process because of their numerous capabilities. Learners might not make use of these capabilities and fail to profit from them. One core genAI affordance is to adapt any learning material to the learner’s individual needs. To help learners make use of these adaptation capabilities, instructional prompts can help, but they can also have negative effects on cognitive load. This study investigated K-12 learners’ perceptions and interaction patterns with a chatbot, focusing on their use of content adaptation capabilities. In this experimental study with 106 secondary school students, a group receiving adaptation guidance (including brief instruction, prompt suggestions, and adjustable chatbot response length and language level) was compared to a control group without adaptation guidance. Results show that learners perceived chatbots as easy to use despite limited prior experience. Without guidance, they underutilized the chatbot’s adaptation capabilities. The experimental group used twice as many adaptation prompts (M = 6.0) compared to the control group (M = 2.5), without experiencing increased cognitive load. Interestingly, both groups showed similar knowledge gains and reported high satisfaction levels. These findings suggest that easily implementable interventions can enhance students’ use of genAI chatbot capabilities, potentially improving their self-regulated learning experiences. Future research should explore the long-term effects of adaptation guidance on learning outcomes and self-regulated learning skills.