Developing Students’ HOTS Digital Literacy Profile by Using IRT

This study aimed to develop and validate a digital literacy higher-order thinking skills (HOTS) cognitive test for engineering students at Universitas Negeri Yogyakarta in Indonesia. Utilising a quantitative research design, the study involved 95 participants and employed expert judgement to assess content validity through the Aiken V Index, which yielded a high average validity score of 1.783. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) confirmed a single factor structure with strong fit indices (CFI = 0.98, TLI = 0.97, RMSEA = 0.08), indicating robust construct validity. Reliability analysis revealed a high internal consistency (α = 0.92). Item Response Theory (IRT) calibration demonstrated satisfactory model fit, with moderate discrimination and average difficulty indices for the test items. The findings affirm that the developed test is both reliable and valid, effectively capturing students' digital literacy HOTS profiles, thereby providing valuable insights for educational interventions aimed at enhancing digital literacy skills in engineering education.