The effects of problem format, number of independent variables, and their interaction on student performance on a control of variables reasoning problem

Abstract This study was designed to investigate the effects of problem format and the number of independent variables in the problem on the responses of students to a control of variables reasoning task. The Bending Rods problem was used. A four times two factorial research design with four levels (2‐, 3‐, 4‐, 5‐variables) of the number of independent variables and two formats (essay and multiple choice) was set up for the investigation. 548 eighth graders in 24 science classes in a suburban public school district northwest of Chicago participated in the study. Students were administered the Bending Rods problem in groups in their science classes. Each classroom was randomly assigned to a cell in the research design. Initial analysis of the data indicated four separate levels (2‐, 3‐, 4‐, 5‐variables) of the independent variable number of independent variables in the problem' were unnecessary. The 2‐ and 3‐variable levels were pooled as were the 4‐ and 5‐variable levels to give two distinct levels of this variable. Analysis of variance indicated that: (1) task format had no effect on subject's scores; but (2) the differences between subjects' mean scores on the 2‐, 3‐variable essay versions and the 4‐, 5‐variable essay versions are significantly greater than the mean scores of corresponding multiple choice versions of the task, which exhibit rather uniform scores; and (3) the 2‐, 3‐variable forms together were significantly less difficult for students than were the 4‐, 5‐variable forms together. Discussion focuses on interpretations that include: (1) task demands on working memory; (2) Pascual‐Leone's M‐energy; and (3) the use of testwiseness strategies in solving essay and multiple choice tasks, respectively. Implications for science teachers are set forth in view of these discussion points and their relevance to test construction and classroom assessment of learning objectives in science.