Towards An Unruly World: Ideas Of Interest (Day 1 -- International Security Forum)

Related Posts:
Live-blogging The ISF
BBC Monitoring

The ISF Conference began yesterday at the International Conference Centre in Geneva. It is a magnificent facility, with 640 attendees, and the speaker's list looks very interesting but the rules of the conference are going to cramp my style.

The whole ISF is being held under the Chatham House Rule and this prohibits me from citing who said what. That said, I can report some of the ideas to emerge from the conference.

Some of these ideas were unsurprising but some of them either had surprising twists or were brand new ideas entirely. I will start with the unsurprising ones first:

The Parade of Horribles. Climate change, war, terrorism, energy, food and water shortages, the demographic time bomb, WMDs, cyberwar, economic collapse... Lawyers call a lengthy string of terrible things like this the "Parade of Horribles". It is a rhetorical device designed to engage attention and compel action and it was used in much the same way here. What was interesting was the degree of consistency -- the same list of threats came up in multiple different contexts.

China and India. There seemed to be a good bit of concern and a certain sense of inevitability that China and India would emerge as future powerhouses. Nothing new here, of course, but the degree of certainty about this shift in power was noticeable. Certainly I heard more of this type of talk here in 24 hours than you do in the States over the course of several months. In addition, it was rarely just "China" and mostly "China and India".

Solutions. There was also a remarkable consensus about solutions: Reinforce international institutions (including the UN Security Council and regional initiatives); establish the "right amount" of regulation for markets; and increased coordination between defense, diplomacy and development agencies/organizations.

Some of the surprises:

Cyberwar/crime. There was far more talk concerning this issue than I thought there would be. If you take a look at the panels in detail, you can see that cyber takes up quite a bit of real estate. Interestingly, there is not a single panel devoted exclusively to terrorism (though it permeates the discussions here) but there are several devoted exclusively to cybersecurity issues. At one point there was even talk of Alternate Reality Games!

Emphasis on intelligence analysis tools and methods. No one called it that, of course. No, they used code words such as "security foresight" but they couldn't fool me -- they were talking about intelligence. Collection of information did not seem to be an issue; figuring out what the info we have actually means did.

"Resiliency". I think I witnessed the birth of a buzzword. The word "resiliency" kept coming up. The need to build resilient societies, have resilient systems. I think that the use here is similar to the way environmentalists use the word "sustainable" and the way network theorists use the word "robust". There was a sense that the runaway process of globalization had sacrificed too much of the world's resiliency for the sake of efficiency and that at least part of the current set of problems was due to this imbalance (Note: John Robb has been talking about the "resilient community" for quite some time but it was interesting to see the same term crop up here).

Part 9 -- Final Thoughts (Evaluating Intelligence)

Part 1 -- Introduction
Part 2 -- A Tale Of Two Weathermen
Part 3 -- A Model For Evaluating Intelligence
Part 4 -- The Problem With Evaluating Intelligence Products
Part 5 -- The Problem With Evaluating The Intelligence Process
Part 6 -- The Decisionmaker's Perspective
Part 7 -- The Iraq WMD Estimate And Other Iraq Pre-War Assessments
Part 8 -- Batting Averages

The purpose of this series of posts was not to rationalize away, in a frenzy of legalese, the obvious failings of the Iraq WMD NIE. Under significant time pressure and operating with what the authors admitted was limited information on key questions, they failed to check their assumptions and saw all of the evidence as confirming an existing conceptual framework (While it should be noted that this conceptual framework was shared by virtually everyone else, the authors do not get a free pass on this either. Testing assumptions and understanding the dangers of overly rigid conceptual models is Intel Analysis 101).

On the other hand, if the focus of inquiry is just a bit broader, to include the two ICAs about Iraq completed by at least some of the same people, using many of the same processes, the picture becomes much brighter. When evaluators consider the three documents together, the analysts seem to track pretty well with historical norms and leadership expectations. Like the good weatherman in Part 2 of this series, it is difficult to see how they got it "wrong".

Moreover, the failure by evaluators to look at intelligence successes as well as intelligence failures and to examine them for where the analysts were actually good or bad (vs. where the analysts were merely lucky or unlucky), is a recipe for turmoil. Imagine a football coach who only watched game film when the team lost and ignored lessons from when the team won. This is clearly stupid but it is very close to what happens to the intelligence community. From the Hoover Commission to today, so-called intelligence failures get investigated while intelligence successes get, well, nothing.

The intelligence community, in the past, has done itself no favors for when the investigations do inevitably come, however. The lack of clarity and consistency in the estimative language used in these documents made coming to any sort of conclusion about the veracity of product or process far more difficult than it needed to be. While I do not expect that other investigators would come to startlingly different conclusions than mine, I would expect there to be areas where we would disagree -- perhaps strongly -- due to different interpretations of the same language. This is not in the intelligence community's interest as it creates the impression that the analysts are "just guessing".

Finally, there appears to be one more lesson to be learned from an examination of these three documents. Beyond the scope of evaluating intelligence, it goes to the heart of what intelligence is and what role it serves in a policy debate.

In the days before the vote to go to war, the Iraq NIE clearly answered the question it had been asked, albeit in a predictable way (so predictable, in fact, that few in Washington bother to read it). The Iraq ICAs, on the other hand, come out in January, 2003, two months before the start of the war. They are generated in response to a request from the Director of Policy Planning at the State Department and are intended, as are all ICAs, for lower level policymakers. These reports quite accurately -- as it turns out -- predict the tremendous difficulties should the eventual solution (of the several available to the policymakers at the time) to the problem of Saddam's Hussein's WMDs be war.

What if all three documents had come out at the same time and had all been NIEs? There does not appear to be, from the record, any reason why they could not have been issued simultaneously. The Senate Subcommittee states on page 2 of its report that there was no special collection involved in the ICAs, that it was "not an issue well-suited to intelligence collection." The report went on to state, "Analysts based their judgments primarily on regional and country expertise, historical evidence and," significantly, in light of this series of posts, "analytic tradecraft." In short, open sources and sound analytic processes. Time was of the essence, of course, but it is clear from the record that the information necessary to write the reports was already in the analyst's heads.

It is hard to imagine that such a trio of documents would not have significantly altered the debate in Washington. The outcome might still have been war, but the ability of policymakers to dodge their fair share of he blame would have been severely limited. In the end, it is perhaps the best practice for intelligence to answer not only those questions it is asked but also those questions it should have been asked.