Friday, 7 November 2008

Egg on faces

dewey_defeats_truman_lg I have to laugh every time pundits make predictions, as if they know the future, and as if 'the polls' were always accurate.   Every person making predictions should study Harry Truman's famous victory in 1948 and John Major's victory in 1992 if they think polls are always a reliable guide to what voters are going to do, or if pollsters ever admit they got it wrong.

Because what a punter tells a pollster doesn't guarantee that's what a punter will do on the only Saturday that counts.

MajorDickhead And Colin Espiner points out about modern polling that when "they're calling thousands of people to find the thousand they need," you have to wonder about their accuracy.  But this doesn't stop the pundits like Duncan the Braindead and Guyon the Hand-Wringer making out they can.  Ironically at a time when they're perhaps less reliable than they've ever been, these two and others like them base almost all of their 'analysis' on what these polls might 'indicate' every week -- which means their analysis tends to change every week -- which really means their analysis is worthless, particularly when tiny movements in support for the minor parties will have major effects on who finally gets to govern.

And there's something else about analysis that's almost solely based on 'what the polls say,' which is that what the polls don't tell you these days is all the 'Don't Know' voters, which are still a goodly number, or the punters whose responses were discarded, or all the support for the non-parliamentary parties (which has ranged from 1 to 3% over the campaign period), which makes it a little hard for these non-parliamentary parties to take off.  Because polls themselves are somewhat self-selecting when people start to take them seriously.  Voters see 'the polls' and make a decision based on what the polls say, a thousand of whom tell the pollsters what they've decided based on earlier polls, and then the polls tell you what those thousand punters decided based on those earlier polls.

Anyway, since pundits only look at the latest polls, this is really to say to all to say those pundits that 1) to rely on last night's polls as an indicator of what's going to happen tomorrow would be foolish, particularly given the importance of the minor parties, and 2) it was damned irresponsible of people like Duncan the Braindead to tell punters thirty-six hours before they vote that these polls were a prediction of what those very punters were going to do.


  1. Yes polling has its problems, which is why we also use markets to to make predictions. iPredict is New Zealand's first real money prediction market. iPredict currently has PM.national at $0.8498 and PM.Labour at $0.1873.

  2. If we had a market in NZ polling accuracy, I'd be shorting them.

    Some people point to the bias produced by telephone polling to explain the 'Truman upset" (they say that more republicans owned phones, and that the pollsters didn't adjust for the fact).

    Our pollsters rely on telephone surveys. Have they factored in the fact that a lot of younger voters don't have landlines?


1. Commenters are welcome and invited.
2. All comments are moderated. Off-topic grandstanding, spam, and gibberish will be ignored. Tu quoque will be moderated.
3. Read the post before you comment. Challenge facts, but don't simply ignore them.
4. Use a name. If it's important enough to say, it's important enough to put a name to.
5. Above all: Act with honour. Say what you mean, and mean what you say.