Reports, politics, and intelligence failures: The case of Iraq

Abstract The intelligence failure concerning Iraqi weapons of mass destruction (WMD) has been the center of political controversy and official investigations in three countries. This article reviews the Report on the U.S. Intelligence Community's Prewar Intelligence Assessments on Iraq, Senate Select Committee on Intelligence, 7 July 2004, Review of Intelligence on Weapons of Mass Destruction, a Report of a Committee of Privy Councillors to the House of Commons, 14 July 2004 (the Butler Report), Report to the President of the United States, The Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction, 31 March 2005. It explores the reasons for their deficiencies and the failure itself. This case and the investigations of it are similar to many previous ones. The investigations are marred by political bias and excessive hindsight. Neither the investigations nor contemporary intelligence on Iraqi WMD followed good social science practices. The comparative method was not utilized, confirmation bias was rampant, alternative hypotheses were not tested, and negative evidence was ignored. Although the opportunities to do better are many, the prospects for adequate reform are dim. Key Words: intelligenceintelligence failuresIraqi WMDofficial inquiries Acknowledgement I would like to thank Richard Betts, Peter Gourevitch, Deborah Larson, Melvyn Leffler, Rose McDermott, Paul Pillar, Marc Trachtenberg, James Wirtz, and several members of the intelligence community for ideas and comments. Notes 1Truth in reviewing requires me to say that I chair the CIA's Historical Review Panel which advises the Director on declassification policies and priorities, wrote a post-mortem for the CIA on why it was slow to see that the Shah of Iran might fall that located a number of errors which recurred in the Iraq case ('Analysis of NFAC's Performance on Iran's Domestic Crisis, Mid-1977–November Citation1978', declassified as CIA-RDP86B00269R001100110003-4), and led a small team that analyzed the lessons of the Iraq WMD failure. This essay has been cleared by the CIA's Publications Review Board, but nothing was deleted and there is nothing of substance I would have added if I had not had to submit it. 2The Report of the Inquiry into Australian Agencies, Canberra, July Citation2004 (the Flood Report) is not as detailed as the US and UK reports and I will say little about it. The UK House of Commons Foreign Affairs Committee and the Intelligence and Security Committee had investigations and reports, although what is of value in them for our purposes is subsumed by the Butler Report. The UK also held a special investigation into the suicide of David Kelly and the related question of whether the British government had 'sexed up' its public dossier on WMD (the Hutton Report). The Butler Report covers some issues of policy as well as intelligence, in part because in the UK the line between the two is not as sharply drawn as in the US. Indeed, 'assessment is really viewed in the UK as a government function and not specifically an intelligence function': (Philip Davies, 'A Critical Look at Britain's Spy Machinery', Studies in Intelligence, 49/4 (Citation2005), 41–54). For other analyses of the Butler Report, see Philip Davies, 'Intelligence Culture and Intelligence Failure in Britain and the United States', Cambridge Review of International Affairs 17 (Oct. Citation2004), 495–520; Nigel West, 'UK's Not Quite So Secret Services', International Journal of Intelligence and CounterIntelligence 18/2 (Spring Citation2005), 23–30; Mark Phythian, 'Still a Matter of Trust: Post-9/11 British Intelligence and Political Culture', ibid. 18 (Winter Citation2005–Citation2006), 653–81; Alex Danchev, 'The Reckoning: Official Inquiries and The Iraq War', Intelligence and National Security 19/3 (Autumn Citation2004), 436–66 Prime Minister Blair gave his response to the Butler Report in a speech to the House of Commons on 13 July 2004. For Central Intelligence Agency (CIA) responses, see Associated Press, 'CIA Revising Pre-Invasion Iraq Arms Intel', New York Times, 2 Feb. 2005; CIA Directorate of Intelligence, 'Continuous Learning in the DI: May 2004 Review of Analytic Tradecraft Fundamentals', Sherman Kent School, CIA, Tradecraft Review 1 (Aug. Citation2004); Richard Kerr et al., 'Issues for the US Intelligence Community', Studies in Intelligence 49/3 (Citation2005), 47–54. For earlier discussions of the intelligence failures, see Peter Bamford, Pretext for War: 9/11, Iraq, and the Abuse of America's Intelligence Agencies (New York: Doubleday Citation2004) and, especially, John Prados, Hoodwinked: The Documents that Reveal How Bush Sold Us a War (New York: New Press Citation2004). These accounts do little to explain the failures, however. It also appears that intelligence made errors in other areas, especially in underestimating the deterioration of Iraq's infrastructure. But little attention has been focused here, or on areas of intelligence success, especially in anticipating the obstacles to political reconstruction. 3Danchev, 'Reckoning', 437. 4The four investigations of Pearl Harbor conducted in the five years after it failed to settle the basic questions, as shown by Martin Melosi, In the Shadow of Pearl Harbor: Political Controversy over the Surprise Attack, 1941–46 (College Station: Texas A Vaughan draws in part on Charles Perrow, Normal Accidents with High Risk Technologies (New York: Basic Books Citation1984); for another superb analysis of this type see Scott Snook, Friendly Fire: The Accidental Shootdown of U.S. Black Hawks Over Northern Iraq (Princeton UP Citation2000). 6Steven Weisman and Douglas Jehl, 'Estimate Revised on When Iran Could Make Nuclear Bomb', New York Times, 2 Aug. Citation2005. 7Much of this literature is summarized in Timothy Wilson, Strangers to Ourselves: Discovering the Adaptive Unconscious (Cambridge, MA: Harvard UP Citation2002). For a further discussion of this and related issues see Robert Jervis, 'Understanding Beliefs', Political Psychology, forthcoming). 8For a good argument that intelligence mattered less in the Cold War than is generally believed, see John Lewis Gaddis, 'Intelligence, Espionage, and Cold War History', Diplomatic History 13/2 (Spring Citation1989), 191–212; for the general (and overstated) claim that intelligence matters little in warfare, see John Keegan, Intelligence in War (London: Hutchinson Citation2003). For a small but important case in which good intelligence derived from intercepted cables guided policy, see Ken Kotani, 'Could Japan Read Allied Signal Traffic? Japanese Codebreaking and the Advance into French Indo-China, September 1940', Intelligence and National Security 20 (June Citation2005), 304–20. Not only may policy be independent of intelligence, which may not have been the case in Iraq, but good policy may rest on bad intelligence. In the most important case of this kind, in prevailing on his colleagues to continue fighting Nazi Germany in June 1940, Winston Churchill utilized estimates of German strength that were even more faulty than the WMD estimates: David Reynolds, 'Churchill and the British 'Decision' to Fight on in 1940: Right Policy, Wrong Reasons', in Richard Langhorne (ed.), Diplomacy and Intelligence During the Second World War, Cambridge, UK: Cambridge 147–67. 9Numbers 13: 1–2, 31–32; for the most recent report of an intelligence failure, see Bill Gertz, 'Analysts Missed Chinese Buildup', Washington Times, 9 June 2005. 10The literature is enormous: the best discussion is Richard Betts, Surprise Attack (Washington DC: Brookings Institution Citation1982); the classic study is Wohlstetter, Pearl Harbor, see also Emphrain Kam, Surprise Attack: The Victim's Perspective (Cambridge, MA: Harvard UP Citation1988). Good historical studies are Ernest May, ed., Knowing One's Enemies: Intelligence Assessment Before the Two World Wars (Princeton UP Citation1984) and the special issue of Intelligence and National Security 13/1 (Spring Citation1998) edited by Martin S. Alexander on 'Knowing Your Friends: Intelligence Inside Alliances and Coalitions from 1914 to the Cold War', also in book form that year, now available from Routledge. For a detailed study of the failure of American, Dutch, and UN intelligence to anticipate the capture of Srebrenica and the massacre of the men captured there, see Cees Wiebes, Intelligence and the War in Bosnia, 1992–1995 (Munster: Lit Citation2003). Much of this work rests on analysis of how individuals process information and see the world, as I have discussed in Perception and Misperception in International Politics (Princeton UP Citation1976). For an application of this approach to improving intelligence, see Richards Heuer, Psychology of Intelligence Analysis (Washington DC: CIA Center for the Study of Intelligence Citation1999). For a superb study of individual differences in accuracy of predictions and willingness of change one's mind, see Philip Tetlock, Expert Political Judgment (Princeton UP Citation2005). In my post-mortem on why CIA was slow to see that the Shah of Iran might fall (note 1), I came to the conclusion that many of the problems centered on organizational habits, culture, and incentives, however. For all their weaknesses in this area, democracies probably do a better job of assessing their adversaries than do non-democracies: Ralph White, 'Why Aggressors Lose' Political Psychology 11/2 (June Citation1990), 227–42; Dan Reiter and Allan Stam, Democracies at War (Princeton UP Citation2002). 11Of course, it would be difficult to determine the percentage of cases in which intelligence was right or wrong, even leaving aside the questionable nature of such a dichotomy. Indeed, probably the more interesting metric would be a comparison of the success rate of the IC with that of informed observers who lack access to classified information. 12Carl von Clausewitz, On War, ed. and translated by Michael Howard and Peter Paret (Princeton UP Citation1976), 117. The Butler Report uses this quotation as its headnote. 13The Israeli service is often help up as a model, but for a review of its errors, see Ephraim Kahana, 'Analyzing Israel's Intelligence Failures', International Journal of Intelligence and CounterIntelligence 18/2 (Summer Citation2005), 262–79. 14For the cogent but politically unacceptable argument that 'if the September 11 and Iraq failures teach us anything, it is that we need to lower our expectations of what intelligence analysis can…do', see Thomas Mahnken, 'Spies and Bureaucrats: Getting Intelligence Right', Public Interest No.81 (Spring Citation2005), 41. This would mean trying to design policies that are not likely to fail disastrously if the supporting intelligence is incorrect. 15Bob Woodward, Plan of Attack (New York: Simon Jessica Mathews and Jeff Miller, 'A Tale of Two Intelligence Estimates', Carnegie Endowment for International Peace, 31 March Citation2004; Donald Kennedy, 'Intelligence Science: Reverse Peer Review?' Science 303, 26 March Citation2004; Center for American Progress, 'Neglecting Intelligence, Ignoring Warnings', 28 Jan. Citation2004, . One of the main recommendations of the Butler Report was that the Joint Intelligence Committee (JIC) not issue public estimates which, contrary to precedent, it did in this case. 18WMD Commission, 50. 19WMD Commission, 50. 20Butler Report, 13. 21Israeli intelligence did employ a Red Team, but its arguments were found to be unpersuasive: Kahana, 'Analyzing Israel's Intelligence Failures', 273–4. This serves as a good reminder that many of the prescriptions offered in the report would not have changed the outcome. In fact, academic research casts doubt on the efficacy of this approach: Charlan Nemeth, Keith Brown and John Rogers, 'Devil's Advocate Versus Authentic Dissent: Stimulating Quality and Control', European Journal of Social Psychology 31 (Nov./Dec. Citation2001), 707–20. Within CIA, the best work on the related approach of Alternative Analysis: see especially his exposition of how this method could have been used before the Soviet missiles were discovered in Cuba: 'Alternative Analysis and the Perils of Estimating: Analyst-Friendly Approaches', unpublished MS, 6 Oct. Citation2003. 22SSCI, 20–21, 106. 23'Iran: Intelligence Failure or Policy Stalemate?' Working Group Report No.1. 24Only a few scattered individuals dissented. According to Hans Blix, France's President Jacques Chirac was one of them, remarking on the propensity of intelligence services to 'intoxicate each other': Hans Blix, Disarming Iraq (New York: Pantheon Citation2004) 129. 25For a brief discussion of an intelligence success, see David Robarge, 'Getting It Right: CIA Analysis of the 1967 Arab-Israeli War', Studies in Intelligence 49/1 (Citation2005), 1–8. For a discussion of some of the earlier CIA post-mortems, see Douglas Shyrock, 'The Intelligence Community Post-Mortem Program, 1973–1975', Studies in Intelligence 21 (Fall Citation1997), 15–22; also see Woodrow Kuhns, 'Intelligence Failures: Forecasting and the Lessons of Epistemology', in Richard Betts and Thomas Mahnken, eds. Paradoxes of Strategic Intelligence (London: Frank Cass Citation2003), 80–100; John Hedley, 'Learning from Intelligence Failures', International Journal of Intelligence and CounterIntelligence 18 (Fall Citation2005), 435–50. Douglas MacEachin, a former career CIA official, has done a series of excellent post-mortems: The Final Months of War With Japan: Signals Intelligence, U.S. Invasion Planning, and the A-Bomb Decision (Washington DC: CIA Center for the Study of Intelligence Citation1998); Predicting the Soviet Invasion of Afghanistan: The Intelligence Community's Record (Washington DC: CIA Center for the Study of Intelligence Citation2002); U.S. Intelligence and the Confrontation in Poland, 1980–1981 (University Park: Pennsylvania State UP Citation2002). 26In fact, as the NIE was being written, confirming reports were received from a very well placed source. This was so sensitive that it was not shared with the analysts and so did not effect the estimate, but it reinforced the confidence of those in charge of the exercise and of the top policy-makers: WMD Commission, 117. 27Richard Betts, 'Warning Dilemmas: Normal Theory vs. Exceptional Theory', Orbis 26 (Winter Citation1983), 828–33. 28SSCI, 18. 29Irving Janis, Groupthink: Psychological Studies of Policy Decisions and Fiascoes, 2nd ed. (Boston: Houghton Mifflin Citation1983); for later research in this area see Paul 't Hart, Eric Stern and Bengt Sundelius, eds., Beyond Groupthink: Political Group Dynamics and Foreign Policy-Making (Ann Arbor: Univ. of Michigan Press Citation1997). 30WMD Commission, 183. 31For reports of pressures to conform within CIA, see WMD Commission, 191–94; for the argument that the State Department's Bureau of Intelligence and Research (INR) has developed a culture that encourages dissent and the CIA has not, see Justin Rood, 'Analyze This', Washington Monthly (Jan./Feb. Citation2005), 18–21. 32SSCI, 22. In the mid-1980s a similar conclusion was reached by CIA's Senior Review Panel based on examining a number of cases from 1945 to 1978: Willis Armstrong et al., 'The Hazards of Single-Outcome Forecasting', originally in Studies in Intelligence 28 (Fall Citation1984) and declassified in H. Bradford Westerfield, Inside CIA's Private World: Declassified Articles from the Agency's Internal Journal, 1955–1992 (New Haven, CT: Yale UP Citation1995), 238–54. Political psychologists have similarly argued that much information is ordinarily processed 'online', i.e., that as new information is received it is melded with the person's standing judgment on the subject, with the person not being aware of how the latter was formed. See, for example, Kathleen McGraw and Milton Lodge, 'Review Essay: Political Information Processing', Political Communication 13 (Jan.–March Citation1996), 131–38; Charles Taber, 'Information Processing and Public Opinion,' in David Sears, Leonie Huddy, and Robert Jervis, eds., Oxford Handboook of Political Psychology (New York: Oxford UP Citation2003), 433–76. An interesting possible case is the CIA's over-estimate of the time it would take the USSR to produce an atomic bomb. It was so sure that the USSR suffered from a great shortage of uranium that it missed the signs that large-scale enrichment was underway: Donald Steury, 'Dissecting Soviet Analysis, 1946–50: How the CIA Missed Stalin's Bomb', Studies in Intelligence 49/1 (Citation2005), 24–25. 33SSCI, 161–62. 34In addition to the Duelfer Report, see James 'The for CIA Missed in Iraqi Arms New York Times, 26 Jan. 2004. the general is that there are an number of that for of Weapons of Mass Center Report 20 Citation2004), It also not help a CIA is to a case, or by not the reports, but the intelligence that the Perception and Misperception in International a see Social of (Cambridge, MA: Press also see Commission, James Iraqi Arms U.S. New York Times, 6 July For the of negative evidence that was received in another case, see and the German Invasion of (New Haven, CT: Yale UP to the that lack of evidence that Iran was weapons had not in my by that of that a is not How that are not a bad in Weisman and New to Nuclear New York Times, I will in the the of evidence be by the and an argument made by the US in this case as well as about Bill Gertz, Report Iran to Nuclear Washington Times, 2005. 'Spies and For a discussion of the of negative evidence from intelligence in the of see Robert and The of (Winter available at estimates are on that be and in most of cases analysts and fail to For example, as in Iran in intelligence that if it were really the Shah would and the that did not do so was as evidence that the in In and most British that would not and that some of the signs from the German than that only the could for example, WMD Commission, 437. a discussion of similar weaknesses in the British see Butler Report, and (note in the process did about WMD Commission, The most detailed discussion of is and John U.S. the of Times, 20 Citation2005. Commission, 437. This is focused on but reports that CIA other information as and other are not likely to be more after is & Intelligence National No.81 (Fall Citation2005), also Commission, SSCI, For some of the of between analysts and see on a 28 June Foreign Policy Research Commission, For the claim that the House was for political led to the of information on Iran and the of political see the from an Iran to the Director of Intelligence, 2 in John How CIA Analysis the UP of WMD Commission, see discussion of problems on other issues in SSCI, and Miller, and the of CIA Times, 2 of John former Director of Central Intelligence, in see H. Bradford Westerfield, CIA International Journal of Intelligence and CounterIntelligence 9 (Winter Westerfield, CIA (Spring Citation1997), Richard Betts, of and in Betts and (note a of some but also is (note analysis that the that had Although this one of the failure of US to a for WMD as there been of WMD there would have been a that would have into the of America's I explain this failure, but the rest of the US to Jehl, New York Times, March Report, which that the Joint Intelligence Committee (JIC) not have the report in its and in the dossier what it was to for related US intelligence, see SSCI, of July from to David which is in many for New York Review of 9 June a of the about such see the Carnegie 2 Citation2005. in comparison between the of services shed light on the claim that was by surprise by because of the of his intelligence although to be in light of the that Soviet and British estimates were the and David Assessment on Iraq of New York Times, 28 2004. some but a see SSCI, also see WMD Commission, The British Intelligence and Nazi UP The literature on bias is discussed and to in Robert Jervis, Richard and Psychology and UP report the Blair government of the that it intelligence the that analysts were by their of what the government to in New York Review of March 2004, For a case of bias in see Frank to The for Cold (Princeton UP the to which the US intelligence to and the Butler Report, the lack of often of on intelligence, that the was that members of the IC was and trying to his judgment as to how to this in the of one of his top Jehl, Reveal New York Times, and Jehl, On ibid. May it is much for the of the or the to that what or is is a judgment from that of the because it would the question of whether the Commission, SSCI, the in the UK: Butler Report, of the discussions of weapons and also this SSCI, Commission, also see and SSCI, a brief but see the Butler Report, for discussion of a similar issue in the evidence of the of a to be see James or of New York Times, Aug. The of (Cambridge, MA: Harvard UP Citation2003), brief of see SSCI, Butler Report, Commission, this was also in Australian Flood Report, for an War: British Intelligence Assessments and the War Journal of Strategic Studies Citation2004) for other see (New York: Citation2005), Predicting the Soviet Invasion of in before was in of the because a would to an by and so not it would not be in The Wrong (New York: New York Books Citation1977), 106. Report of the to the on Iraq's 2004 Duelfer Report) on Strategic In the American of intelligence in at the and had I what was to take it was so that I probably would have been to it to would the his which was his to be and in Doubleday Citation1976), Report, and are to John had earlier that on the were by his of to the Foreign Affairs Citation2004), Report, also The Duelfer Report not be In many it like a of much information remains and there is some between this report and (note and would not have and it is not which of them was viewed as most and The UN for the latter to continue even after the former and had in This a because if had been the main have his at