Can We Generalize from Student Experiments to the Real World in Political Science, Military Affairs, and International Relations?

The authors conducted an experiment with a group of military officers and replicated it with a group of students at a public university in the United States. The experimental scenario dealt with a decision problem in the area of counterterrorism. The authors found that while more than one-third of students recommended doing nothing, the overwhelming majority of military officers (more than 90 percent) recommended doing something. Also, military officers exhibited less maximizing and more satisfacing decision making than students. The results show that relying on experiments with students “playing” the role of real-world national security policy makers may bias the results. The two groups are, in fact, very different. Based on student samples, it is possible to accept propositions that would not be found with samples of elite decision makers and reject propositions that may be right. However, it is possible that students can be assigned to experiments where they represent the “public” and not elites.