Violent Killings of Emos in Iraq

I wish this was an Onion story.

If your culture is frightened of Weezer fans, well, your culture is weak, and should be ashamed of its cowardice.

Who you hate says a lot about you. Nazis, snakes, OK.

But, seriously, hardline clerics, Emos?

Is Using a Minotaur to Gore Detainees a Form of Torture?

Serious discussion of a serious issue. Some say the minotaur issue has been exaggerated by the media.

Margins of Error in Iraq

Let me start by saying that finding out what is actually happening in Iraq is very difficult, and that actually gathering that information requires admirable courage.

That said, I think that those that try and make sense of what’s going on there need to display at least a minimal level of introspection, humility, and recognition of how limited our information is.

In September 2007, the British polling survey firm ORB issued a survey from Iraq that suggested that more than a million Iraqi citizens had died violently since the invasion. The ORB website used a less neutral term: “murdered”.

ORB’s core competency seems to be the familiar western opinion survey by random phone interview. That isn’t actually very relevant to doing a cluster sample mortality survey in a war zone. And the survey work itself was done by an Iraqi firm, IIACSS, that didn’t exist before 2003, founded by an Iraqi with apparently only limited formal training in survey methodology.

This may not be as much of an issue in surveying public opinion. Similar opinions may be broadly spread through either the Iraqi population as a whole or regional subsets. If so, exactly where you sample may not affect results too seriously.

Violent mortality, on the other hand, can be lumpy. It’s pretty clear that some parts of Iraq are a lot more violent than others, and even when you get down to the local neighborhood some blocks or even houses can be can be lot luckier than others, or the reverse. Mortality surveys are very sensitive to even small sampling errors.

ORB doesn’t seem to have provided a lot of added value in terms of oversight and quality control. They originally reported that the survey was based on “a nationally representative sample.” Later, they admitted that the original survey was “undertaken in primarily urban locations” Given that about a third of Iraqis live in rural areas, this is a significant omission, and ORB failed to disclose the choice when the results were first published. They should have. When they did a follow up survey and sampled more rural areas, they reduced their initial estimate of violent deaths by about 200,000.

Their latest press release on the study indicates that it covered "112 unique sampling points". That is, it was a cluster sample of the sort used by other researchers in Iraq. It wasn’t the 2,163 independent observations you’d get if you did that number of random phone interviews. ORB calculated their margin of error as though it was that number of independent observations, not the much larger margin of error for a cluster sample of that number of households. This not only grossly understates the level of uncertainty in the estimate, but makes you wonder how well they understand this sort of work.

And even when theoretical sampling error is correctly calculated, that doesn’t include other sources of uncertainty. Researchers make subjective decisions on when to skip areas because of security or other issues, and when to do follow up interviews of clusters missed for these or other reasons, violating a truly random sample. Survey teams may curbstone, or invent responses, particularly when the risks of carrying out the survey are as real as they are in Iraq. There may be errors in tabulating the data.

The survey was originally published with a glaring error in Baghdad’s religious composition undetected. This does not speak well for ORB’s diligence in checking for possible error, bias or fraud.

Collecting the number, age and sex of household members is a powerful tool for checking the plausibility of the sample in a mortality survey. It is unfortunate that neither the ORB survey nor Burnham et al did so, since the absence of this data limits the credibility of the information gathered at considerable personal risk by the survey teams.

Follow up visits by supervisors is another powerful check of survey accuracy. I think it’s important for those publishing survey results in Iraq to disclose if this was done, as well as the number of sampling points and what factors, if any, were used in weighting the raw data.

How Bad Is It in Iraq? (IV)

A study based on the Iraq Family Health Survey (IFHS) and recently published in the New England Journal of Medicine sheds new light on violent deaths in Iraq. It estimates that violent deaths are 2-4 times higher than the tally of civilian deaths collected from media reports by Iraq Body Count (IBC). This disparity is unsurprising, since that tally attempts to screen out combatant deaths, and media reports will miss some deaths. The study period ended in June of 2006: applying the same ratio to the current IBC tally would give a death toll in the 150,000-300,000 range.

This fits with the earlier ILCS survey, which estimated violent deaths 2-3 times the contemporary IBC count. This estimate based on cemetery traffic suggests a ratio in the 2.5-4.5 times range.

The study by Burnham et al published in The Lancet estimated violent deaths 10-20 times higher than IBC. There’s an obvious conflict between Burnham et al and IFHS. IFHS had a larger sample size, more resources and better supervision. Both studies failed to survey some of the planned clusters: 11% in IFHS, 6% in Burnham et al. IFHS made an effort to compensate for the missed clusters, Burnham et al did not. IFHS also made an effort to reflect regional population changes from migration during the study period. Burnham et al did not.

Some supporters of Burnham et al are still defending that study. One argument they make is that IFHS isn’t so different if you measure “excess death,” that is the increase in death rates, including nonviolent deaths, over pre-war conditions. I don’t think this works: the IFHS authors didn’t try to calculate that and argued, I think correctly, that recall was worse for the pre-war period. Certainly the recalled death rate for that period was low compared to neighboring countries. Subtracting the pre-invasion death-rate from the post invasion rate could give a spurious increase because of recall issues.

Supporters of Burham et al also complain that the IFHS annual death rate does not show the strong increase from 2003 to 2006 recorded in other sources. However, the range of sampling error is substantial for the annual figures, and the difference in the IFHS trendline and that shown by IBC is not statistically significant.

One of the strengths of the IFHS data is that it also looked at other demographic data, and the large sample size narrowed the margin of error. If Burnham et al was closer to the truth about violent deaths than IFHS, the result should be visible in the IFHS demographic data. If 2.5% of the population is being killed in an armed conflict, (as Burnham et al claim) and most of those deaths are military-age males (one of the few points on which Burnham et al and IBC agree), then the result should be a strong male/female imbalance in adult Iraqi demographics.

The predicted imbalance does not occur in the IFHS data, except for the cohort that was unfortunate enough to reach fighting age back when Saddam Hussein was invading his neighbors.

How Bad is it in Iraq? (III)

The enormous cemetery at Najaf is a favored burial site for Shiites from all over Iraq, and provides macabre evidence for the state of affairs in that country.

Dhurgham Majed al Malik, 48, whose family has arranged burial services for generations, said that this spring, private cars and taxis with caskets lashed to their roofs arrived at a rate of 6,500 a month. Now it's 4,000 or less, he said.

Malik said that the daily tide of cars bearing coffins has been a barometer of Iraq's violence for years. The number of burials rose and fell several times during Saddam Hussein's persecution of Shiites, and it soared again during the eight years of the Iran - Iraq war in the 1980s.

Then in the 1990s, the daily average fell to 150 or less, Malik said. With the current war, the burials again reached 300 daily.

In the early days of the war, some bodies brought for burial had been victims of Saddam, found by their families in unmarked mass graves. Later, there were surges; September 2005 marked a high point after a stampede during a Shiite Muslim festival killed hundreds on a Baghdad bridge. More than 1,300 were buried in a single day, Malik said.

If the preinvasion burial rate was up to 150 a day, and estimates of the preinvasion annual death rate of about 6 per thousand are reasonably correct, then roughly 1/3 of Iraqi dead (and 2/3 of Shiite dead) were being buried there. The increase over the preinvasion rate would be a strong indicator of the level of violent deaths since then.

Malik reported an increase over preinvasion rates of 150 burials a day at their peak, presumably some months in 2006. That would have included unidentified dead from Baghdad. Unidentified dead from Baghdad could exceed three dozen a month pre-invasion, climbed to 140 a month in July of 2005, and peaked at 2000 a month during the worst periods after the Askariya mosque bombing in February 2006. Many of those unidentified dead would have been Sunni: perhaps half or more. It seems likely that Baghdad was not the only place that sent unidentified dead to Najaf. An official reported that 40,000 unidentified bodies have been buried there since the invasion, and the unidentified dead reported from Baghdad would only account for about 2/3 of that total. Some number of the remainder would have been Sunni as well.

When violence was at its peak, then, the cemetery would have received about 2,700-3,300 extra Shiite corpses in the worst months, over and above the normal preinvasion burials, and all or most dead from violence or other consequences of the conflict there. If the prewar burial pattern held, about half that number of additional Shiites killed by violence would have been buried elsewhere.

Surveys have given various estimates of the violent death rate for Sunni Arabs, ranging from about the same per household or family rate as Shiites to twice as high. The same surveys find Kurdish violent deaths per family or household ranging from 1/3 to 1/6 of the Shiite rate. Given their share of the population and assuming similar household and family demographics, deaths from Sunni Arabs and Kurds might have totaled from 1.2 to 2.5 times the number of Shiite burials in Najaf, and total violent Iraqi deaths would have equaled 2.7 to 4 times that number, or 7,000-13,000 a month.

In the spring of 2007 Malik reported 6,500 private burials a month. In addition, about 300 unidentified bodies a month were delivered by truck from Baghdad to Najaf, with a like number going to a new cemetery at Karbala, and perhaps additional unidentified dead from elsewhwere. Applying the same multiples would give a national violent death rate of 6,000-10,000 a month.

These estimates are 2.4-4.6 times the Iraq Body Count tally for civilian deaths in media reports for the same periods. This disparity is unsurprising, since media reports would miss some deaths and total deaths would include a significant number of combatants who were not civilians. There was a similar ratio of 2.7/1 between the ILCS demographic survey, which asked about total war-related deaths in Aril 2004, and the IBC civilian tally for the same period.

Applying these multiples to the IBC tally to date would suggest a total violent death toll of 200,000-300,000.

If the above seems too complicated, let me give you a simplified version that sets a crude upper limit on violent deaths in Iraq.

Pre-invasion, based on reports from cemetery workers at Najaf, it looks like about 2/3 of Iraqi Shiite dead were buried at Najaf. Conditions are more chaotic now, but the relative political and economic position of Shiites has improved, so it seems plausible that a similar or higher proportion holds.

In 2006, up to 2000 unidentified dead from Baghdad were buried at Najaf each month. Many of these would have been Sunni Arabs, so counting all post-invasion burials at Najaf as Shiite would significantly overcount the Shiite death rate. However, let us simplify and ignore this factor.

At their highest, burials at Najaf were 300 a day, 150 higher than the pre-invasion rate. The likeliest explanation for the disparity is an increase in violent deaths post-invasion.

The highest survey-based estimate of Sunni Arab deaths per household is twice the Shiite rate, and most surveys give a lower ratio. It appears that Sunni Arabs are about 75% as numerous as Shiites in Iraq. The highest survey based estimate for Kurdish deaths per household is 1/3 the Shiite rate. Iraqi Kurds seem to be about 1/3 as numerous as Shiite Arabs.

We then have the following upper bound for monthly violent deaths at their peak:

4,500 Shiites buried at Najaf.
2,250 Shiites buried elsewhere.
10,125 Sunni Arabs
750 Sunni Kurds
17,625 total.

This is 5.8 times the highest IBC monthly total. Applying the same multiple to the average of maximum and minimum IBC totals to date gives about 460,000. Since this ignores the problem of double counting unidentified dead at Najaf, and uses the highest surveyed value for Sunni Arab deaths it is likely to be a considerable overestimate.

ORB’s September Survey and Baghdad Deaths

ORB has quietly revised the religion data in their survey of Iraq released earlier in September, but there are still strange things about the Baghdad results.

The gender demographics for Baghdad still seem to conflict with the reported violent death rate. 421 households, the weighted number for Baghdad, would have had about 2750 members in 2003, if their size was typical of Iraq, with about 700 adult males and a like number of adult females. According to both media reports collected by Iraq Body Count and those polls that asked the question, adult males represent about 90% of the violent deaths in Iraq since the invasion. It is plausible to estimate that of the 295 violent deaths reported in the weighted ORB poll, about 250 were adult males (18 or older), 15 were adult females, and the rest were children. The adult survivors would then be about 450 males and 685 females: 40% male and 60% female. Children coming of age since 2003 might reduce the imbalance slightly: to 42%/58%, assuming no sexual disparity in deaths among those who were 14-17 at the time of the invasion. This last is not necessarily a plausible assumption: 14-17 year old males are probably at higher risk than females of the same age.

ORB reported that their adult Baghdad respondents were 49% male. Plausible extrapolation of the reported violent death rate above, and the age and gender of reported victims of violent death in Iraq, suggest that given that death rate Baghdad adults should only be 40-42% male. That’s a significant conflict

There’s another reason to distrust the Baghdad death rate in the September ORB poll. In a poll they released in March, they asked a similar but broader question: had the respondent had a relative murdered in the past three years? 26% said yes: 31% said so in Baghdad, and 24 % in the rest of the country. In the September poll, 12% outside Baghdad has lost at least one household member. If we assume that the Iraq Body Count was a fairly consistent undercount during the preceding six months and use it to estimate the change in cumulative violent deaths, that would have been 10% at the time of the March survey. Outside Baghdad, the two polls seem in reasonable agreement: it seems entirely plausible that an extended family in Iraq is about 2.5 times the size of the immediate household.

But extrapolating the March Baghdad responses forward on the same basis predicts a household death rate of 15%: higher than the rest of Iraq, but less than a third of that reported for Baghdad in the September poll. Alternatively, projecting the September Baghdad results backwards implies that Baghdad extended families are only 25% larger than the immediate household living under one roof. That seems highly implausible.

Why Democracies Fail

A paper by Andrew Enterline and J. Michael Greig looks at the history of imposed democracies and the implications for Iraq and Afghanistan. I don’t think it necessarily helps to look only at that narrow subset of new democracies; I think the risk factors are similar for indigenous ones like the first French Republic. In any case, as Steve Muhlberger points out, Enterline and Greig’s criteria seem a bit daft: Canada and New Zealand imposed democracies? I don’t think so.

The paper looks at Polity IIId rankings as a tool to track the states. Eyeballing those ratings, some factors seem plausible to me.

How close to liberal democracy were previous regimes over the preceding hundred years, including not just elections, but an independent judiciary, rule of law, strong property rights, etc.? If the previous regime was colonial rule, how democratic was the ruling state, and how much local autonomy did the colony have? Much of the Philippines’ colonial history was under relatively illiberal Spanish control, while India got over a century of British rule. I think that mattered.

How much of the local economy is based on the extraction of resources like gold or oil? If it is, a predatory government can have a pleasant lifestyle while not paying a lot of attention to the general welfare of the citizens.

Finally, does the country have a meritocratic civil service, and is that institution well established? I don’t think this factor is generally given the importance it deserves. When civil service jobs are distributed by competitive written examination, on a model that goes back to Imperial China, they cease to be spoils to be fought over, and the incumbents have a vested interest in keeping it that way. (The ability of the exams to measure anything more than the candidates' fitness to write about, say, classical poetry seems to be largely secondary.) And a meritocratic civil service can exist regardless of whether or not the current regime is democratic.

A meritocratic civil service does seem to be a common factor in many of the new democracies that survived the transition from autocracy or colonial rule: countries as otherwise disparate as Germany, Japan, India, Malaysia, South Korea, Taiwan and Turkey.

Baghdad Now 60% Christian

ORB, a British polling group, has recently released a poll they took in Iraq in August that suggest that there have been more than a million Iraqi violent deaths because of the invasion, (and about 800,000 of those in Baghdad, based on 206 interviews in that city.)

This is newsworthy if true, and ORB has issued a press release to that effect. They’ve missed an even bigger story. Based on the same 206 interviews, in theory selected entirely at random from the Baghdad population of about six million, Baghdad is mostly Christian: 37% Orthodox, 13% Catholic, 9% Protestant and 1% Christian (page 46).

One possible explanation is that the ORB interviews were entirely random, and that the Baghdad population is actually about 60% Christian, plus or minus random sampling error.

Another, which I prefer, is that it is very difficult to do a true random sample when your next planned interview is on the other side of a checkpoint manned by heavily armed locals with a dim view of outsiders who want to ask possibly inconvenient questions.

This would explain another puzzle in the ORB results. They imply that almost one in two Baghdad households have lost a family member. Both media reports and previous polls have indicated that the Iraqis being killed are overwhelmingly adult males: they are both more likely to be targeted and more likely to be exposed to attack. Iraq Body Count estimates that 90% of the civilian deaths are adult males, and including soldiers and insurgents would make the ratio even more extreme. Iraq is also a young country, with about half the population under eighteen. One would expect that if the Baghdad households sampled are like typical Iraqi households, those deaths would significantly impact the ratio of male to female adults answering the poll. This does not occur in the Baghdad poll results.

It seems likely that something decidedly unrandom has happened to the Baghdad sample. Or some Baghdad respondents are using an expansive definition of household that includes, say, everybody in their apartment building. Or both. Or there was an error in coding the results.

Update: ORB issued revised data, dated 9/20/2007, that gives changed results for all religion categories in Baghdad while leaving other data unchanged. Christians are now reduced to 3% of the city’s population. Although no explanation has been offered on their website, it seems likely that many of the Baghdad responses on religion were originally entered incorrectly.

The gender demographics for Baghdad still seem to conflict with the reported violent death rate.

How Bad Is It in Iraq? (II)

ORB, a British polling group, has just released the results of a poll it took in Iraq in August that suggests that over a million Iraqis have died violently since the US invasion. However, it’s important to understand the level of uncertainty involved in the estimate. It is very, very hard to quantify what’s happening in a place as chaotic as Iraq.

One measure of the level of uncertainty is that the weighted estimate of violent death is about 50% higher than the raw figures would suggest. ORB asked Iraqis a number of questions, including whether one or more people living under their roof had been killed violently since the invasion. If the sample is randomly chosen, the margin of error should be fairly predictable.

One of the key challenges is a random sample. You can pick phone numbers at random: this works well if almost everyone has a phone, and is equally likely to answer it. If not, you can adjust for the difference between those that answer phones and those that don’t, if you can.

Or you can send out polling teams in a way that insures that everyone has about the same chance of being polled. This requires that you have a good idea of who is living where. This can work well under quite bad conditions. If a large share of the population is living in squalid refugee camps, and you have a good idea how many are living in each camp because a relief agency is handing out rations, and the residents have been thoroughly randomized by panicked flight from murderous militias, you can get a good random sample this way, at least for the people in the camps.

Iraq fits neither condition. Having completed their initial survey, ORB concluded they needed to massage the numbers. Did they undersample Baghdad? If so, they should adjust those numbers, assuming they have a good estimate of the current total population, and whether their sample was proportionate to the current population of the neighborhoods they sampled.

There’s a dilemma here. If the government is functioning adequately, you don’t estimate violent deaths by survey: you ask it to tabulate the corpses in the morgues, or the ration card holders that have stopped eating. On the other hand, if it has lost count of the corpses, then they are probably even less capable of telling you the current population of a specific neighborhood or region net of massive refugee flows.

Massaging polling numbers is a tricky business. Your poll may capture a surprisingly low number of Sunni Arab adult males. This might be sampling error, best dealt with by re-weighting the sample. On the other hand, it may reflect the fact that a large number of the expected Sunni Arab adult males are, in fact, dead. If that’s the case, re-weighting the sample will only distort the truth.

Something along those lines seems to be reflected in the recent ORB poll. Looking at the raw, un-weighted responses, 7% of Iraqi Kurds have lost a household member to violence since the invasion, and 6% of those that identify themselves as Shiites. Many Iraqis refused to claim a particular sect for the pollsters, and simply called themselves Muslims. 9% of these lost a household member. This group included some Kurds, and backing those out at the Kurdish death rate suggests that Arabs that identified themselves as “Muslim” rather than a particular sect had a household violent death rate of about 10%. This group includes both Sunni and Shiite Iraqi Arabs.

The group that identified themselves as “Sunni” had a 32% household violent death rate in the un-weighted ORB poll. However, most Kurds are Sunni, but reported much less violence. Backing them out at the average rate of reported violence for Kurds suggests that Sunni Arabs reported household violent death since the invasion at about 38%.
If the poll results are remotely in the right ballpark, Arabs that identify themselves as Sunni are suffering much worse losses than other groups in Iraq: the sort of casualties that knocked Russia and the Central Powers out of World War I, and drove the French army to the edge of mutiny.

Since the poll makes no effort to isolate civilian casualties, one plausible explanation of at least part of the disparity is that Sunni insurgents are being killed in large numbers.

While this poll will be taken by some as confirmation of Burnham et al (2006) published in the Lancet, they do contradict each other in one important sense. According to Burnham et al, Baghdad’s level of violence was about average for Iraq. According to ORB, Baghdad is about twice as deadly as Iraq as a whole.

No Touch of Harry in Iraq?

It’s been reported that Prince Harry will not be going to Iraq. I can appreciate the reasons for that, particularly the risk the presence of such a high value target will create for his immediate comrades. But I think the British army is missing an opportunity.

There’s a time honored tactic used by the British monarchy to deal with just this sort of problem, going back at least to the battle of Shrewsbury: scatter a number of doubles about the battlefield in copies of the royal armor. In 1403 that was plate harness and coat armor bearing Quarterly France Ancient and England rather than a Scimitar light tank with “Windsor Rules!” painted on the turret, but the principle is the same. A dozen doubles could be pre-deployed in widely scattered locations, and then revealed and concealed to give the illusion of rapid movement about the theatre of operations. For added entertainment value, his theoretical itinerary from A to B to C and back to A could be leaked, with doubles making appearances at the destination points only to confirm the illusion and bait traps. The intervening roads would be populated only by ordinary traffic, frustrated insurgents, and over-flying hellfire-armed drones and helicopter gunships.

They seek him here, they seek him there
Jihadis seek him everywhere
Is he in Basra or in Baghdad?
That damned elusive Windsor lad

How Bad Is It in Iraq?

This is not an easy question to answer. There’s good reason to think the official statistics don’t capture the full death toll: a country that’s gone through invasion, a radical regime change, insurgency and deadly sectarian conflict may have other priorities than rigorously counting all violent deaths and forwarding the information to the central government. Totaling casualties reported in the English-language media can only capture a lower bound: not all casualties are going to be captured in this way.

Polling samples have their own problems. When the misery is unevenly distributed, oversampling the worst or the best regions can badly distort the results. Eliminating this problem requires reasonably accurate regional population estimates. With hundreds of thousands of Iraqis either fleeing abroad or displaced within Iraq and a government struggling to perform its most basic functions, how accurate are such estimates likely to be?

With those caveats, two recent polls give us a clearer picture of conditions in Iraq: one carried out for ABC News, USA Today, the BBC and ARD German TV by D3 Systems, the other by ORB. Both sampled a large number of locations throughout Iraq.

In the ABC poll, 17% reported that a family member living in their household had been physically harmed by the “violence that is occurring in the country at this time”. The ILCS survey indicated an average Iraqi household size of 6.6, so this implies a minimum average casualty rate of 2.6%. Iraq Body Count compiles English–language media reports of violence in Iraq, and says that 37% of reported civilian casualties to date are fatalities.

Media reports are probably more likely to underreport injuries than fatalities. On the other hand, some families may suffer more than one casualty. If casualties were distributed purely randomly this wouldn’t affect the overall rate much, but they aren’t. If one family member is targeted or living in a risky area, there is an increased chance that others are at risk as well. If the two unknowns roughly cancel each other out, that would imply violent deaths that were about 1% of the population: far lower than Burnham et al, but still a terrible human tragedy.

Baghdad is more deadly than the country as a whole: D3 said 77% reported they had had a friend or family member harmed in Baghdad, compared to 52% overall and 29% in Kurdistan. ORB reported that 51% of those surveyed in Baghdad had had a relative, friend or colleague murdered, compared to 38% in Iraq overall

42% told D3 they thought the country was in a civil war. Given a more nuanced range of options in the ORB study, 27% said the country was in a state of civil war, 22% that the country was close to civil war but not there yet.

D3 reported 42% thought they were better off now than before the war, 36% worse. In the ORB survey, 49% thought they were better off now, 26% that they were better off under the previous regime.

D3 said 48% thought the invasion was absolutely or somewhat right, 52% absolutely or somewhat wrong.

They overwhelmingly wanted the US to leave, but only 35% wanted the US to leave immediately

How Many Sunnis?

A recent poll carried out in Iraq for ABC News, USA Today, the BBC and ARD German TV had one result that seems to have gotten less media coverage than it deserves. Conventional wisdom has been that Sunni Arabs represent something like 12-22% of the population of Iraq.

Recent survey data, including this poll, have had different results. This survey found 47 percent Shiite Arabs, 35 percent Sunni Arabs, 15 percent Kurds and three percent others.

D3 Systems reports that in its previous surveys it has seen Shiite Arabs in a range from the high 40s to low 50s, and Sunni Arabs in a range from the high 20s to mid-30s. The 35 percent Sunni Arab estimate in this poll is at the high end of its previous data, but within that range. This poll had more sampling points than any previous individual national study in Iraq by D3/KARL

Another large poll, carried by ORB gave similar results: 32% Sunni Arab, 42% Shiite Arab, and another 9% Arabs who didn’t identify their sect. It turns out that the lower estimates, while widely quoted, don’t seem to have a lot of sourced evidence behind them.

As far as we have been able to ascertain there is no official Iraqi estimate of the country's Sunni vs. Shiite Arab populations, and no single authoritative source of empirical data on the subject.

Needless to say, if these numbers are correct and previous Western estimates wrong, they should radically alter our view of likely outcomes in a sectarian conflict in Iraq. If Sunni Arabs there have a population that is 75% of the Shiite population rather than 35%, the decision of insurgents to provoke sectarian conflict begins to seem much more rational.

2006 Study on Iraqi Deaths Contradicts 2004 Conclusions by Same Authors

The controversial Johns Hopkins study recently published in the Lancet follows up on a 2004 study by many of the same authors. The 2004 study was criticized because the limited sample size meant that the death estimates had a broad range of uncertainty. Nonetheless, the authors concluded that:

In the period of the study, about 60,00 Iraqis died from violence that wouldn't have happened if the prewar status quo had continued.

Most of them were killed by the invading coalition, and most of the victims were women and children.

In addition, another 40,000 Iraqis died from nonviolent causes that would have lived if the prewar status quo had continued, because of post invasion economic disruption, dislocation of the health care system, and so on.

In 2006, the team published a new study with a much larger sample size. The new study suggests that during the period measured by the 2004 study:

About 110,00 Iraqis died by violence. Most were killed either by insurgents or unidentified gunmen. Of the remainder killed by the coalition, most were males of military age.

Nonviolent deaths did not increase in the period immediately following the invasion. On the contrary, they decreased.

In a kinder, gentler universe, we might expect Les Roberts to explain:

"Wow. We really got the 2004 study wrong. In retrospect we should have understood that we didn't have enough data to know if most of the dead Iraqis were killed by the coalition or by local talent. We thought it was obvious that the coalition was using excessive force. In retrospect, we should have asked the coalition to be more agressive than it was. We were wrong and we're sorry"

I don't think we live in such a universe.


Will McLean

More than 600,000 killed in Iraq?

So says a recent Johns Hopkins study published in the Lancet. Is it true? That's a huge number: somewhat more, as a share of population than the American Civil War, which took longer, and was fought with mass conscripted armies and 19th c. medicine and sanitation.

It's important to point out that this isn't an estimate of civilian dead. The study makes clear that there was no effort to exclude combatants from the death toll, and the study indicates that military age males account for a majority of violent deaths. The New York Times incorrectly described it as an estimate of civilian deaths, and I have not yet seen them publish a correction.

The Lancet's editor does seem to be to be a strong advocate for a particular political point of view with an interest in seeing the invasion of Iraq proved to have been a Very Bad Thing. That may have affected the timing of the study's release, and how vigorously it was reviewed before publication. Ultimately, however, the conclusions will rise and fall on the quality of the study itself.

The vast discrepancy between the estimate and the official figures raises concern, but not necessarily a fatal one. It is possible that local tallies of violent deaths at hospitals and mortuaries are either not making it in to the central government or being suppressed. That they are not being recorded at all seems an insufficient explanation: about 90% of the households in the survey that reported a death were able to provide a death certificate when asked.

More troubling is the discrepancy between the mid 2004 UNDP survey and this one. It used similar cluster sampling methodology to the Johns Hopkins study, but estimated 25,000 killed in the year following the invasion, compared to about 90,000 estimated by the Johns Hopkins study in the same period. It would appear that the UNDP figure was not intended to capture ordinary criminal murders, but that's hardly enough to explain the discrepancy: Insurgency, counter insurgency and sectarian violence seems to account for the lion's share of violent deaths.

Why might the John Hopkins study be wrong? To start with, it relies on dividing the country into equal size clusters, and sampling a group of households in each cluster. Accuracy depends on reasonably accurate estimates of population by region. One challenge is that accurate population estimates are hard to find in Iraq. The last nationwide census was 1993. Further, there's reason to question the accuracy of the Saddam era censuses: the ruling party had strong motives for inflating the number of Sunnis at the expense of other groups. The problem is compounded by the study's use of a two year old estimate.

If the population is overestimated in the more dangerous parts of Iraq, and underestimated in the less, then the more dangerous parts will be overrepresented in the sample. There's good reason to think that this was the case. The Sunni triangle would have benefited from any Saddam era selective distortion, and is generally considered to be one of the more violent parts of Iraq, and indeed this reflected is in the Johns Hopkins study. Further, you would expect the more violent regions to lose population share relative to the rest of Iraq since 2004, both because of flight and higher death rates.

Another problem might be nonrandom sampling of households. As I understand the protocol, the first household in the cluster was randomly chosen, and then the team would go to the nearest neighboring household, and so on. But in a lot of communities, a household will have two neighbors that are equally close. Ideally, the team should flip a coin, but that might not be what happens. "Asking about people killed in the fighting, are you? Then you should visit Widow Tikriti. She's right next door".

How can one confirm or refute the study? I have two suggestions. The first would involve asking local hospitals and mortuaries about the ratio of violent to nonviolent deaths. The other would be looking at demographics. If the study is correct, the violent death rate among adolescent and adult males has been very high since the invasion: 5% among 15-29 year olds, 9.5% among 30-44 year olds, and 7% among 45-59 year olds. Female violent death in the same cohort has been a very small fraction of that. The impact should be very visible in the male-female ratios in the relevant age groups. In doing such a study, one should keep in mind that sex ratios in the 44-59 age groups were already unbalanced by about that amount because of the deaths from Saddam's wars and massacres.

Update: More commentary at Asymmetrical Information, with more elsewhere on the site. And more here and here.