Defining the need: digital literacy skills for first-year university students

Purpose This study aims to build context-specific measurement tools to assess first-year students' digital literacy skills. The purpose is to collect data to provide more meaningful skills support for new students and raise their awareness about the digital literacy skills fundamental to academic study. Design/methodology/approach A set of three scales were developed and piloted at the beginning of 2018 and 2019 fall semesters: (1) a self-assessment survey, (2) an online test to collect indirect evidence and (3) a set of performance tasks. In both rounds, the scales were examined for reliability and construct validity and improved accordingly. The third round, conducted with a different scope, enabled a final reliability check for the online test. Findings The results indicate that first-year students need support with higher-order digital literacy skills. Low performers tend to overestimate their skills, so self-assessment surveys fall short in providing needs assessment but provide evidence of awareness when combined with the other scales. Performance tasks provide direct evidence but are not practical with larger numbers of participants. An online test to deliver indirect evidence works well but requires careful reliability considerations. Research limitations/implications The conceptual framework for digital literacies offered in this study was narrowed down to the immediate skills to maintain a feasible scope. The sample size was small in both rounds, but the findings gave insights regarding the skills under scrutiny. Similarly, the scales were subject to close examination and were continuously improved, and the imperfections were monitored carefully while drawing conclusions. Practical implications All of the three scales are ready to be used. Depending on the purpose, they can be given to first-year students together or separately to define their needs for support. Both versions of the survey and performance tasks are presented in the paper, and the test can be found at this link: Digital Literacy Test for Preview. Originality/value There are three methods to measure digital skills. Although there are studies using individual or paired methods, this study is distinctive as it combines all three. While other scales to measure digital literacy skills exist, they are not appropriate in every context. Therefore, this study will serve as an example for those wishing to devise context-specific scales.