Assessing Algorithmic Thinking: A Psychometric Analysis of Different Assessment Approaches in a Pilot Study

Algorithmic thinking is a key skill in higher education; however, its assessment has been limited by a predominant focus on final outcomes rather tan underlying cognitive processes. This study aimed to analyze the psychometric performance of an instrument designed to assess algorithmic thinking through different performance assessment approaches in Mexican university students with varying levels of ability. A maximum-performance test comprising four assessment approaches was developed and administered to 141 students from two public institutions, using indicators from classical test theory and a qualitative analysis of participants’ perceptions. The results showed that thefunctioning of the assessment approach depended on students’ ability levels and revealed discrepancias between perceived difficulty and observed performance. As a limitation, the pilot nature of the study and the relatively small group sizes are acknowledged. The study provides evidence supporting the need for differentiated assessment approaches. It is concluded that the assessment of algorithmic thinking should be aligned with students’ level of development