Wednesday, October 11, 2006

A Young Bloggers Primer on Intellectual Honesty

I would urge everyone to read the Lancet study that I mentioned below, even if you intend to dismiss it as nothing more than anti-American bilge. You can find the PDF here, and even if you skip the deepest statistical analysis it's relatively clear what they're getting at. Most interesting to me was reading the danger that the survey teams were exposed to - for example, not being able to carry GPS locator units for fear of being mistaken for American agents marking airstrike targets.

More importantly, the next-to-last page of the article lists out all of the reasons the authors could think of for why their estimate could be wrong. They list several reasons why they may have overcounted the dead, and several more reasons they could have undercounted them. It's rare, outside the world of genuine science, to see people concede the possibility of failure. That said, the same page also has this chilling note:
In Iraq, as with other conflicts, civilians bear the consequences of warfare. In the Vietnam war, 3 million civilians died; in the Democratic Republic of the Congo, conflict has been responsible for 3.8 million deaths; and an estimated 200 000 of a total population of 800 000 died in conflict in East Timor. Recent estimates are that 200 000 people have died in Darfur over the past 31 months. We estimate that almost 655 000 people - 2.5% of the population in the study area - have died in Iraq. Although such death rates might be common in times of war, the combination of a long duration and tens of millions of people affected has made this the deadliest international conflict of the 21st century, and should be of grave concern to everyone.
So, having said all this, I would hope that warmongers would do a few things before they decide to slander or dismiss this survey. First, read these two posts (here and here) about the actual math behind the 2004 survey, and the dismantling of the innumerate arguments that the Republicans and their fellow travellers used to try and delegitimize the previous survey.

Secondly, I would like you to obey the following rules of intellectual honesty, or simply concede you're an innumerate hack:

If you intend to simply reject this study out of hand, you may never, ever cite a public opinion poll - for any reason - again in your life. The mathematical laws that give opinion polling their accuracy also apply to this survey. Reject one, reject the other.

If you intend not to reject this study, but instead prefer to believe in the lowest possible estimate of casualties the survey allows (slightly less than 400,000) you are also required to concede that believing in the highest possible estimate of deaths (in the latest survey, over 900,000) is also perfectly acceptable. The margin of error cuts both ways.

If you intend to accuse the mathematically-literate of accepting this survey's results out of a desire to delegitimize the case for war in Iraq, you must also accept that these findings do, in fact, delegitimize the case for war in Iraq. If this level of casualties didn't affect opinion on the war, your charge would be simply false, wouldn't it?

If you cite passive surveys (like the UK-based Iraq Body Count) to argue against this survey, you need to argue that no violent act ever happens anywhere without being reported in major media outlets. Otherwise, you need to concede the possibility - indeed, the certainty - that the IBC and other similar methods are catching only a small fraction of the dead. Generally speaking, the IBC could be catching only one in ten deaths and still be doing extraordinarily well by historical standards. Multiplying the IBC's estimated by ten, incidentally, brings us a number well within the Lancet's confidence interval.

Leave your own suggestions for intellectual honesty in the comments.

3 comments:

Scott Neigh said...

If you intend not to reject this study, but instead prefer to believe in the lowest possible estimate of casualties the survey allows (slightly less than 400,000) you are also required to concede that believing in the highest possible estimate of deaths (in the latest survey, over 900,000) is also perfectly acceptable. The margin of error cuts both ways.

I would also add that if you intend to believe the lowest possible estimate from this study and then try to argue that 400,000 people dying is in some sense qualitatively different from 900,000 people dying, then you should admit that you are an immoral, soulless cretin and you do not deserve to be listened to.

Thanks for posting this analysis and pre-emptive rebuttal of the spin that's sure to come!

Saskboy said...

Worse, this study shows that the previous estimates of ~30K are either flawed, or intentionally wrong. Either this study, or the others are far from the mark and someone screwed up or screwed us over.

M. Simon said...

600,000 over 3 years = 200,000 per year vs the 30K to 40K actually reported.

A factor of 5 difference would have been noticed.

I think the study guys used an incorrect sampling method.