Thursday, 25 August 2011

The failure of forecasting, part XVII [updated]

I've posted before on the failure of economic forecasters to do what they purport to do best, i.e., forecasting the future (see here, here and here, for example.)

For all their high-tech mathematical models however, when it comes to it these alleged forecasters are as blind as an English referee.  Why? Because all their models do is spit out information that extrapolates existing trends, and suggests that things will always tend back to the mean.

Which is not only fatuous, it doesn’t work. Especially at times like these.

Just look at recent evidence gathered by Orewa-based commentator Rodney Dickens using NZIER research [Reviewing the Consensus Forecasts for economic growth - pdf], which compares predictions-against-performance for the so-called “top-ten” economists surveyed each quarter by the NZ Institute of Economic Research—the sort of chaps you see on the news each night being asked by newsreaders to comment sagely on what’s going to happen next. The average predictions by these chumps are called the 'consensus forecasts.'  And they’re decidedly below average.

They all got it wrong predicting GDP.  They all got it wrong predicting residential building activity. They all got it wrong predicting interest rates. They all got it wrong predicting bond yields. And most got it wrong predicting the TWI (but even a monkey on a typewriter could get it right once, right?). So in sum, they were all as wrong as Gareth Morgan’s investment advice—and the fact is they’re always this wrong, almost very time.

Why? Because the future is inherently uncertain, and econometric analysis can’t change that one whit. All it does is provide mathematical justification for going wrong with confidence.

imageNow, it’s true as economist Ludwig Von Mises say that “historians and statisticians content themselves with prices of the past,” while “practical man looks at the prices of the future”--and that econometricians pretend to use the prices of the past to predict the prices and conditions of the future.  But the fact they can’t, and never can, just reinforces the crucial role of entrepreneurs in driving economic activity: in taking risks on the future with their own money based on their own individual estimations of the future.

It’s not blind crystal-ball readers who move the world, it’s entrepreneurs. And most econometricians wouldn’t even know how to spell the word.

UPDATE: Kudos to Matt Nolan at the Visible Hand blog for taking on the challenge of defending forecasting.  His defence, really, rests on two things. He says:

  • “Entrepreneurs  … make the choices, and … they will value this type of service insofar as it helps give them information that allow them to make better informed choices”; and
  • “If it was possible to forecast the sharp rise in fuel prices, the impact of new banking regulations (with no relevant history), the global droughts, the earthquakes in Canterbury, the snow storms during lambing in the South Island, and government policy changes they would have been less wrong.”

Now it’s true that lots of things went wrong last year.  But forecasters’ failures can be seen every year, not just last year.  See what I mean. (Not to mention their signal failure to spot either the Crash of 2008 or the slide that started this latest Depression within a Depression Recession within a Recession.

And while it’s certainly true just as Matt says that entrepreneurs take advice from all over, including from forecasters (as Ludwig Von Mises also observed* in the link I posted above) that doesn’t mean they should listen to every charlatan that comes through their inbox. 

Frankly, what’s the point if they’re consistently and abjectly wrong?

* * * * *

* Ludwig Von Mises on “The Place of Economics in Learning”:

“In fact reasonable businessmen are fully aware of the uncertainty of the future. They realize that the economists do not dispense any reliable information about things to come and that all that they provide is interpretation of statistical data referring to the past. For the capitalists and entrepreneurs the economists' opinions about the future count only as questionable conjectures. They are skeptical and not easily fooled. But as they quite correctly believe that it is useful to know all the data which could possibly have any relevance for their affairs, they subscribe to the newspapers and periodicals publishing the forecasts. Anxious not to neglect any source of information available, big business employs staffs of economists and statisticians".”
”It is obvious [however] that men are fallible, and businessmen are certainly not free from this human weakness. But one should not forget that on the market a process of selection is in continual operation. There prevails an unceasing tendency to weed out the less efficient entrepreneurs, that is, those who fail in their endeavors to anticipate correctly the future demands of the consumers.”


  1. As a individual that uses econometric models on a daily basis to model microeconomic relationships to uncover profit opportunities, I would like to clarify the distinction between the econometric models to which the author refers - macroeconomic models incorporating flawed aggregate proxy variables - and microeconometric models, which make use of directly observed measures. With the latter, we boast greater than 90% out-of-sample forecast accuracy, which can be leveraged to help with the decision making process by the business.

  2. "It’s not blind crystal-ball readers who move the world, it’s entrepreneurs.."

    All hail the mighty Hotchins and Murdoch, for they are our saviours! Imagine if they went on strike! The horror!

  3. Bevan said...
    I would like to clarify the distinction between the econometric models to which the author refers - macroeconomic models incorporating flawed aggregate proxy variables - and microeconometric models, which make use of directly observed measures

    Macro-economic theory is a direct translation of the micro-economic theory, which is wrong. If the Macro is wrong which is based on micro theory, ie, a single rational representative agent, being applied to the large scale (macro), then I can't see how macro can be wrong while micro is supposed to be correct.

  4. "Future Babble - why expert predictions fail, and why we believe" by Dan Gardner is a book worth reading.
    Gives many reasons why we mere mortals should take certain futures rather lightly.


Say what you mean, and mean what you say.

(Off-topic grandstanding and trolling is moderated. If it's not entertaining.)