Do 45% of College Students Lack Critical Thinking Skills? Revisiting a Central Conclusion of Academically Adrift

The educational literature, the popular press, and educated laypeople have all echoed a conclusion from the book Academically Adrift by Richard Arum and Josipa Roksa (which has now become received wisdom), namely, that 45% of college students showed no significant gains in critical thinking skills. Similar results were reported by Pascarella, Blaich, Martin, and Hanson after the publication of Arum and Roksa's book in 2011. However, these authors' statistical tests were conducted incorrectly, and therefore this 45% finding is fundamentally untrue. We demonstrate that a correct statistical analysis would have found that far fewer students show significant gains in critical thinking. However, this does not reflect on student learning; instead, it reflects on how hard it is to find a statistically significant result when assessing student change on a student‐by‐student basis. This article discusses valid methods for testing the significance of gain scores of individual students.