The Peradex: a performance-based assessment approach for adaptive expertise using design scenarios and artificial intelligence

Higher education has the responsibility to educate students to take up grand societal challenges, such as climate change or threats to global health. To this end, students and professionals need to develop adaptive expertise (AE), which is the ability to perform at a high level when facing new or changing problem situations. Scholars struggle with measurement of AE. Various questionnaires that measure AE are self-assessments, which might lead to bias. We currently lack an unbiased assessment tool for AE. A promising direction lies in the development of design scenarios, presenting real-world, open-ended problems that can be solved. In this paper, we developed an assessment approach called the Peradex (Performance-based Adaptive Expertise). The approach is based on a series of 72 design scenarios using artificial intelligence (AI). Next, we asked a panel of 535 students and young professionals to propose solutions to the problems presented in four of these design scenarios that were presented to them. We used Phi-4, a local large language model, to assess to what extent the solutions demonstrated AE. We conclude that the Peradex is reliable and valid for formative assessment purposes.