A Brief Note on Statistics

Yesterday, regarding the Lancet survey of casualties in Iraq, I noted that rejecting this survey out of hand - a properly randomized survey with a large sample size - effectively requires you to reject every single public opinion poll ever done. Nevertheless, the talking point among all innumerate rightists seems to be that the methodology is "flawed" somehow. Nevermind that no specific methodological criticisms have been levelled that pass the laugh test.

Broadly speaking, the right is rejecting this survey based on the fact that it is a survey, not a body-by-body counting. But nobody with any expertise - anywhere - believes that a body-count would seriously produce a more accurate number. Let's illustrate with a historical example: The United States Census Bureau has been counting humans - free and otherwise - longer than there's been an Iraq, or a Canada for that matter. The US Constitution requires that the Census be taken every 10 years and that Americans be counted individually. However, leading up to the 2000 census, there was a minor controversy over whether the Census bureau would be allowed to use statistical sampling to estimate where the discrepancies lay between the official tally (required by the Constitution to apportion seats in Congress) and reality.

Nobody in either party denied that statistical methods would provide a better, more accurate picture of where Americans lived and what kind of people they are. But head counts systematically undercount minorities and the poor, while overcounting the wealthy whites. Guess which party wins and loses in this state of affairs? Guess which party controls Congress? Guess which party claims the majority of the Supreme Court vote? The Supreme Court ruled - correctly, I hasten to say - that the letter of the Constitution is very clear, and statistical sampling could not be used to apportion seats, but could be used for other purposes.

The point is this: The Census Bureau - with more continuous experience doing head counts than any other institution on Earth - believes that statistical sampling is a more accurate way to collect data about a large population than a simple head count.

Statistical surveys are, in a very real way, the foundation of the modern technocratic state. GDP, unemployment, inflation, wages, all the numbers that make up the headlines to our lives are based on statistics - collected in much the same way as this Iraqi casualty survey was taken. The authors of the Lancet study have been open and honest in all of their methods, their data, their assumptions, and the potential faults to their conclusions. Methodologically, this survey is as sound as any other - or else the Lancet wouldn't have risked it's considerable reputation to publish it, especially after the 2004 survey.

If the pro-war crowd can't stomach the results of their splendid little war, maybe - just maybe - they should have thought a bit more about it beforehand.

Just remember - 92% of the Iraqis were able, when asked, to provide valid physical documentation of the deaths they claimed. Every single objection to this survey is resting its hopes on the remaining 8%.