Requirements Monitoring and Diagnosis for Improving Adaptive E-Learning Systems Design

Aim/Purpose: In this paper, we highlight the need to monitor and diagnose adaptive e-learning systems requirements at runtime to develop a better understanding of their behavior during learning activities and improve their design. Our focus is to reveal which learning requirements the adaptive system is satisfying while still evolving and to provide specific recommendations regarding what actions should be taken and which relevant features are needed to help meet the specified learning requirements. Background: Adaptive e-learning systems research has long focused on user modeling and social learning to personalize each learner experience, while fewer instruments are reported to assess the quality of the solutions provided by such adaptive systems and to investigate their design problems. The design problems may emerge due to ever-evolving requirements being statically specified at design stages and to the changing environments that can be difficult to control and observe. The combination of some or all of these factors can lead to a definition of inconsistent or insufficient adaptation rules, which in turn may prevent these systems from providing appropriate resources to learners even if the needed ones have been accounted for within the knowledge space. Methodology: An empirical study has been performed to check and validate the behavior of a real-world adaptive e-learning system under four stated requirements. The study used a novel monitoring and diagnosing tool that reads the collected data from the system and checks its behavior against constraints that are derived automatically from the requirements specification. Contribution: The results provide statistical insights and highlight some issues related to requirements compliance at runtime, which helped us detect unforeseen instructional design issues. Recommendations for Practitioners: The study suggests that diagnosing requirements compliance at runtime can be an essential means to increase the confidence about their adaptive e-learning systems capabilities at runtime. Recommendation for Researchers: The study suggests that further research for developing specific indicators related to requirements compliance is needed in the field of adaptive e-learning systems. Future Research: Future work includes the study of possible improvement of our diagnostic tool using probabilistic reasoning.