Development of a digital literacy measurement tool for middle and high school students in the context of scientific practice

Abstract Digital literacy is essential for scientific literacy in a digital world. Although the NGSS Practices include many activities that require digital literacy, most studies have examined digital literacy from a generic perspective rather than a curricular context. This study aimed to develop a self-report tool to measure elements of digital literacy among middle and high school students in the context of science practice. Using Messick's validity framework, Rasch analysis was conducted to ensure the tool's validity. Initial items were developed from the NGSS, KSES, and other countries' curricula and related research literature. The final 38 items were expertly reviewed by scientists and applied to 1194 students for statistical analysis. The results indicated that the tool could be divided into five dimensions of digital literacy in the context of science practice: collecting and recording data, analyzing and interpreting (statistics), analyzing and interpreting (tools), generating conclusions, and sharing and presenting. Item fit and reliability were analyzed. The study found that most items did not show significant gender or school level differences, but scores increased with grade level. Boys tended to perform better than girls, and this difference did not change with grade level. Analysis and Interpretation (Tools) showed the largest differences across school levels. The developed measurement tool suggests that digital literacy in the context of science practice is distinct from generic digital literacy, requiring a multi-contextual approach to teaching. Furthermore, the gender gap was evident in all areas and did not decrease with higher school levels, particularly in STEM-related items like math and computational languages, indicating a need for focused education for girls. The tool developed in this study can serve as a baseline for teachers to identify students' levels and for students to set learning goals. It provides information on how digital literacy can be taught within a curricular context.