Examining measurement properties of AI literacy instruments in the era of generative artificial intelligence: a review study

As generative artificial intelligence (GenAI) becomes embedded in interactive learning environments, measurement of AI literacy is central to instructional design and policy. Valid and reliable instruments are essential for capturing what AI literacy entails in the GenAI era, yet existing reviews have not systematically evaluated the quality of evidence for their measurement properties. This review applied the COSMIN methodology to assess the methodological quality and measurement properties of AI literacy instruments developed in the GenAI context. A search across five databases yielded 37 eligible empirical studies covering four populations, including students, teachers, employees, and the general public. Instrument development was concentrated in East Asian and Western regions and predominantly targeted university students. Self-report scales outnumbered performance-based assessments, with Likert-type formats dominating. The included instruments drew on a range of theoretical sources, with earlier AI literacy concepts often reframed, reorganized, or extended to fit different contexts and emerging assessment needs. Structural validity and internal consistency were the best-supported measurement properties, whereas evidence for several other applicable properties was limited or uncertain. Based on these findings, this review provides guidance for instrument selection according to assessment purpose while considering population and context.