Good “road toll,” bad statistical analysis [updated]
Now that we’ve got some good “road toll” figures for a holiday weekend to shout about, can we try to finally get our heads around the fact that the figures from one weekend—whether good or bad—tell us nothing at all about either the efficacy of road policing policies, changes in driving habits, or improvements in the state of the roads.
Whatever road policing bigwigs might now try to tell you about the success of their ‘zero-tolerance-for-speed’ policy over the weekend, fatality figures from just one weekend are statistically insignificant. Whether good or bad, the sample size itself is so low they tell you nothing at all—not unless you’re a bad journalist keen for a colourful headline, or a traffic policing bureaucrat eager to cement in a new policy. Like Paula Rose, for instance, who “is amazed by what the extra police resources* on the roads and the 'no excuses' speeding policy have done.”
Did their new policy cause the statistics? No, they didn’t. Correlation is not causality, however much the cheerleaders for slow driving might wish it otherwise.
* * * *
* Extra police resources? I’m not sure there were any more “police resources” out this weekend than on any other holiday weekend; and since in driving north from Auckland on Friday night I followed one car with only one headlight and passed another, I doubt that those “resources” were any more vigilant than they normally are. What was your experience?
UPDATE: Zen Tiger does a bit more analysis.