Osama bin Laden masterminded the attack on Manhattan’s World Trade Center on September 11, 2001, and US military preparations for a possible invasion of Iraq began almost immediately thereafter. In February of the following year, Donald Rumsfeld, the United States Secretary of Defense, gave a press briefing. Rumsfeld was asked to comment on reports that no evidence linked Baghdad to terrorist activity. His famous response was widely derided:
“there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know”.[1]
Yet Rumsfeld was saying something important.[2] The follow-up question to Rumsfeld’s musings is less well remembered than the observation that provoked it. The Defense Secretary was asked in which category – known knowns, known unknowns, or unknown unknowns – did intelligence about terrorism and weapons of mass destruction fall? Rumsfeld’s response was “I am not going to say which it is”.[3] But no link between Iraq and the 9/11 attack has been established, and no weapons of mass destruction were found. The Iraq invasion was a military success: US forces rapidly reached Baghdad and toppled Saddam Hussein. But with hindsight the operation was a comprehensive failure of intelligence, judgment, and contingency planning; a failure as politically damaging as the similar failures of intelligence, judgment and contingency planning in the financial sector were economically damaging. In the light of these failures, US government agencies were required to implement a more structured process for providing advice to the President.[4] Analysts were expected to quantify their confidence levels and express them as probabilities.
And so almost a decade later, in spring 2011, President Barack Obama met with his senior security advisers in the White House situation room to consider what he knew would be one of the defining decisions of his presidency. Should he approve the proposed raid by US Navy SEALs on the Abbottabad compound in Pakistan where Osama bin Laden was believed to be hiding? Obama was well aware that a similarly daring plan in 1979 to rescue hostages from the Tehran embassy had ended in fiasco and may have cost Jimmy Carter a second term as President. ’John’, the CIA team leader, was 95% certain that bin Laden was in the compound. But others were less sure. Most placed their probability estimate at about 80%. Some were as low as 40% or even 30%.
The
President summed up the discussion. ‘This
is 50-50. Look guys, this is a flip of
the coin. I can’t base this decision on
the notion that we have any greater certainty than that’.[5] Obama did not mean that the probability that
the man in the compound was bin Laden was 0.5; still less that he planned to
decide by tossing a coin.[6] His summary recognised that he had to make
his decision without knowing whether the terrorist leader was in the compound
or not. Obama would reflect on that
discussion in a subsequent interview: ‘In this situation, what you started
getting was probabilities that disguised uncertainty as opposed to actually providing
you with more useful information.’
[1] US DoD (2002).
[2] We do not, however, attribute the subsequent failures in Iraq to ‘unknown unknowns’ but to a failure to allow sufficient challenge to a prevailing narrative – see chapter 16.
[3] US DoD (2002).
[4] Bowden (2012) p. 160.
[5] Bowden (2012) p. 163.
[6] Friedman and Zeckhauser (2014, p. 2) argue that Obama concluded “that the odds that bin Laden was living in Abbottabad were about 50/50”. We think it is clear from Obama’s further comment that this is not what he meant.
