Measuring knowledge of natural selection: A comparison of the CINS, an open‐response instrument, and an oral interview

Abstract Growing recognition of the central importance of fostering an in‐depth understanding of natural selection has, surprisingly, failed to stimulate work on the development and rigorous evaluation of instruments that measure knowledge of it. We used three different methodological tools, the Conceptual Inventory of Natural Selection (CINS), a modified version of Bishop and Anderson's (Bishop and Anderson 1990 Journal of Research in Science Teaching 27: 415–427) open‐response test that we call the Open Response Instrument (ORI), and an oral interview derived from both instruments, to measure biology majors' understanding of and alternative conceptions about natural selection. We explored how these instruments differentially inform science educators about the knowledge and alternative conceptions their students harbor. Overall, both the CINS and ORI provided excellent replacements for the time‐consuming process of oral interviews and produced comparable measures of key concept diversity and, to a lesser extent, key concept frequency. In contrast, the ORI and CINS produced significantly different measures of both alternative conception diversity and frequency, with the ORI results completely concordant with oral interview results. Our study indicated that revisions of both the CINS and ORI are necessary because of numerous instrument items characterized by low discriminability, high and/or overlapping difficulty, and mismatches with the sample. While our results revealed that both instruments are valid and generally reliable measures of knowledge and alternative conceptions about natural selection, a test combining particular components of both instruments—a modified version of the CINS to test for key concepts, and a modified version of the ORI to assess student alternative conceptions—should be used until a more approprite instrument is developed and rigorously evaluated. © 2008 Wiley Periodicals, Inc. J Res Sci Teach 45: 1131–1160, 2008