Modeling Data From Collaborative Assessments: Learning in Digital Interactive Social Networks

This article summarizes assessment of cognitive skills through collaborative tasks, using field test results from the Assessment and Teaching of 21st Century Skills (ATC21S) project. This project, sponsored by Cisco, Intel, and Microsoft, aims to help educators around the world enable students with the skills to succeed in future career and college goals. In this article, ATC21S collaborative assessments focus on the project's “ICT Literacy—Learning in digital networks” learning progression. The article includes a description of the development of the learning progression, as well as examples and the logic behind the instrument construction. Assessments took place in random pairs of students in a demonstration digital environment. Modeling of results employed unidimensional and multidimensional item response models, with and without random effects for groups. The results indicated that, based on this data set, the models that take group into consideration in both the unidimensional and the multidimensional analyses fit better. However, the group‐level variances were substantially higher than the individual‐level variances. This indicates that a total individual estimate of group plus individual is likely a more informative estimate than individual alone but also that the performances of the pairs dominated the performances of the individuals. Implications are discussed in the results and conclusions.