Problem-based learning design multi-objective optimization to smart education: a digital transformation digital literacy model of higher education

Data-driven approaches of instructional design are more necessary than ever for higher education institutions in the era of digital transformation, especially when faced with noisy and heterogenous educational data. In the digital transformation age, the requirement for data-driven approaches to instructional design that are stable for educational data is growing. Past studies of smart education and problem-based learning (PBL) have been mostly descriptive models or single-dataset studies, with few studies providing evidence for robustness-aware instructional design under different data characteristics. To fill this gap, this study presents a robustness-aware multi-objective optimization method called Cross-Dataset Robust Multi-Objective Optimization (CD-RMO) for exploring PBL design configurations in intelligent learning environments. CD-RMO presents learning gain, development of digital literacy, and implementation cost/risk as three objectives to be optimized simultaneously, and adds robustness to data variability to the objective measurement. The framework is tested on three different heterogeneous datasets related to education: OULAD, EdNet, and KDD Cup 2010, by employing a common feature space and a common PBL design variable. Convergence, diversity, robustness and stability are used as indicators for performance assessment. Under the proposed modeling framework, the results show that CD-RMO yields higher quality Pareto-front, convergence accuracy and robustness consistency compared to the baselines based on experts, classical evolutionary algorithms and a state-of-the-art multi-objective optimizer. In addition, sensitivity and interaction analyses indicate that Scaffolding Strength, PBL Intensity, and Feedback Frequency are significant on robustness-aware optimization performance and other design variables have their effect mostly through interaction effects. Overall, the proposed framework shows that robustness-aware multi-objective optimization is a principled computational methodology to study various trade-offs in ID for intelligent higher education, offering a much more stable and interpretable framework for making decisions in the setting of realistic data uncertainty.