Analyzing Learning “in the Wild”: Modeling Individual Trajectories of Cognitive Engagement via Automated Text Classification and Social Network Analysis

Online citizen science (CS) projects can be seen as informal learning communities where users 'learn in the wild' about the topic at hand, yet assessing such learning through observational studies remains challenging due to the absence of explicit learning goals and formal assessment criteria. To address this, we propose a novel, theory-driven approach that approximates topic-specific learning as the longitudinal evolution of cognitive engagement based on the interactive, constructive, active, passive (ICAP) framework. For this, we fine-tuned a transformer-based language model (DeBERTa-V3-large) to automatically classify forum posts, applying it to a comprehensive dataset of 1.8 million posts across 30 online CS projects. Growth mixture modeling of 471 users with sustained forum activity over their first six months revealed that even among this active minority, the majority exhibited task-oriented routinization, while a small subgroup (8.1%) transitioned from lower to higher modes of cognitive engagement. Social network analysis further revealed that this increase in cognitive engagement is closely linked to social integration, with these users acquiring social capital and being sought out by others. Our findings demonstrate that combining natural language processing with network metrics can provide a scalable method for approximating learning processes in informal settings through cognitive engagement.