How to Construct Achievement Tests to Assess Comprehension

The thesis of this paper is that educational research workers have not yet learned how to develop tests that meet the primitive first requirement for a system of measurement, namely that there is a clear and consistent definition of the things being counted. By achievement I mean a set of questions asked to ascertain what a person has learned from exposure to instruction. There will be in the following a tacit restriction to paper-and-pencil test questions expressed in a natural language, though much of the discussion can be interpreted more broadly. The purpose of this paper is to propose the distinctions necessary to determine whether a person has comprehended an instructional communication, to outline procedures for constructing test questions based on these distinctions, and to show that test questions derived according to the procedures give rise to orderly, sensible data. The starting point for the analysis will be the now-classic Taxonomy of Educational Objectives (Bloom, et al, 1956), and I shall rely on the important recent work of Bormuth (1970) and Hively (1970). Finally, a number of articles in the recent literature will be reviewed to determine how investigators currently construct tests and what they report about these tests.