AI-Powered Multimodal Learning Analytics System Using LLMs for Adaptive & Emotion-Aware Skill Development

AbstractWith the rise in the amount of multimodal data in educational settings, there is a need to harness such data for the improvement of the learning of students. The current learning analytics systems make use of academic and interaction data of students, which may not be sufficient in providing insights into the learning process. This paper proposes LearnSphere AI, an AI-powered multimodal learning analytics system that makes use of visual, audio, and textual learning information with the aid of Large Language Models (LLMs) to offer adaptive and emotion-aware skill development. The proposed learning analytics system is predicated on the principles of utilizing multimodal learner information, learning analytics, learner profiling, AI-assisted interaction, adaptive recommendations, and personalized learning paths. The system will use learning-related parameters such as learning speed, consistency, attention, confidence, engagement, and academic performance to inform adaptation and offer recommendations. Additionally, the system will include an AI companion that will enable learners to interact with the system using natural language to request specific learning assistance. A prototype of the proposed system was developed as a web application to demonstrate the concepts of learner onboarding, personalized dashboards, adaptive recommendations, learning analytics, resource personalization, dynamic learning pathways, and academic reporting. The proposed system shows how multimodal learning analytics and Large Language Models can be integrated into a learning platform to offer a more personalized and adaptive learning experience. The developed prototype serves as a foundation for future work that will seek to utilize real-world multimodal learner data and conduct large-scale educational studies.