Margins of Error in Economic Forecasting.

As someone trained in a real science one of the things that bugs me about the so-called science of economics is that econognomes never cite figures with a margin of error. All measurements have a level of accuracy and precision and a margin of error. If we measure distance then the margin of error is half the smallest unit on our ruler.

Recently two large organisations published growth forecasts for the UK economy. The IMF and Earnst & Young both released figures. Their forecasts for 2013 are 1.4% & 1.6% respectively. Of course the two organisations use different models (though probably both use Neo-classical models) and this means we expect slight differences. But this should not make us any more confident, because we don't know how confident the measure is.

Accuracy in this case will have to be determined in retrospect. If the figure cited is close to actual figure when it is announced then we'd say it is more or less accurate. It would be interesting to see how accurate these kinds of predictions have been over time. This article on the IMF website by Paula Masi suggests that the models are OK for stable conditions, but don't predict changes very well. Which is about the best we could expect for Neo-classical models. Note that in this article IMF forecasts deviated from the real world by ± 1% on average and had to be revised frequently to take account of changing economic policy. This figure of ± 1% seemed to apply quite broadly to other forecasters as well.

The precision of the figure is 1 decimal point in this case. This means that the figure is supposed to be precise to 0.1%. However in a situation where the likely inaccuracy will be ± 1% the extra decimal place is meaningless.

The margin of error is the error inherent in the measurement. For instance if the measures is 1.4% ± 1% then this is an extremely unreliable figure because it could be anything from -0.4% to 2.4%.  If the margin of error is ± 0.1% then the measure is expected to vary from 1.3% to 1.5%. And note that at this level of error the two predictions quoted above overlap, and so we don't treat the difference as very significant. They could both be 1.5% for instance. By comparing the accuracy of forecasts over time we can say that the average error in the prediction is ± 1% and take this for the real margin of error.

In statistical measures the error rate is a factor of the sample size, it's easy to work out. If 5% of the population say they'll vote Green we also know what the expected error is from the sample size. Similarly when comparing two sets of figures statisticians cite the likelihood of a correlation between them. When CERN announced that they had 'found' a new particle, what they actually said was they that had a confidence level of 99.9997% that the CMS detector had found a new boson at  125.3 ± 0.6 GeV/c² within 4.9 σ or a bit over 99.9999% confidence that it wasn't a fluke. Note that it could still be a fluke!

Without this information a figure may as well be plucked out of the air and of course with economic forecasting we suspect that this is exactly what they do! I've complained to the media about the way they cite such figures, and even had journalists agree with me. Citing figures without uncertainty creates a false sense of certainty. But these economic measures are far from certain.At present the margin of error is the same order or magnitude as the measurement. Forecasters, then, could be almost 100% wrong in either direction!

I think it's also pretty clear from all of this that the claim of economics to be a science is far from credible. In fact economic models do make predictions, but they are almost always wrong in ways that should invalidate the theory.

Economists vs Weather Forecasters

Why did god make economists?

To make weather forecasters look good.

Imagine that I am a weather forecaster. I use a mathematical model to predict the weather. But weather is very complex and so I make some assumptions.

Firstly I notice that the total amount of rain is the same as the amount of water put into the atmosphere by evaporation and transpiration. The nett water flow is zero. So I assume that water does not play an active role my model. Similarly wind movements do not change the total amount of air, so I assume that overall it is not important.

In terms of climate I make an arbitrary decision that climate is just big weather. I ignore the way that changes of scale affect how systems behave. And I also assume that weather will tend towards equilibrium that the amount of water in the air, and air movements due to pressure changes will settle down and stay the same unless acted on by an external force (such as an asteroid striking the earth).

The model I produce is a reasonably complex series of algebraic equations that can be handled by a moderately powerful computer.

Would you trust my weather forecasts? Or would you think I was delusional? Would you not point that that when water accumulates it can have dramatic effects such as floods? Would you not point out that localised air movements from tornadoes to hurricanes are actually quite important features of weather? Would you not think that global climate is different from local weather since it covers longer time frames and is on an entirely different scale? And would you not point out that the idea of weather stabilizing at an equilibrium is patently ridiculous, that everyone knows that the weather is inherently unstable and very difficult to predict? Would you not point out that weather and climate modelling can only be done on the worlds most powerful supercomputers and is still far from 100% reliable? Would you not call me a fool?

And economists? Well they have mathematical models too. In these models they see that debt is also credit and cancels out, so it isn't included in their models. They assume that banks are merely passive mediums of money exchange. They assume that the macro-economy is simply a scaled up micro-economy, and that like a simple supply and demand relationship the economy is always seeking equilibrium. In these models consumers are assumed to have perfect knowledge for making decisions, including perfect knowledge of the consequences of their actions, and that masses of people behave no differently from this perfect individual.

It's all patently idiotic. One doesn't have to be an economist to understand that this approach simply cannot work. And it becomes more clear why this approach has not worked. The models will never work. The number of completely insupportable assumptions make it a certainty that they will only ever be right by accident.

Here's Steve Keen explaining it to a group of economists. And note that Unlearning Economics would probably find my analysis overly simplistic: How Not to Criticise Economics.

Now these people, directly or through back seat driving, have repeatedly driven the economy into a ditch. And yet they still clutch the wheel and claim to be doing, and refuse to let anyone else have a go. They seem sincere in their belief that they're continued mayhem is the best that anyone could do under the circumstances. And yet a few marginalised economist who are more willing to ask questions, have made accurate predictions of the present crisis. These people were prescient enough to see the crisis coming, and to write and publish books on the subject. The present crisis was foreseen, it was avoidable, and there is another way. But politicians still have their heads in the sand.
We have to tell our politicians that time is up for economists of the Neo-Classical bent. We need a better model. Better models are available. I have my preferences, but I think the first step is an acknowledgement of failure by the current lot, then a public debate about alternatives, where we can let people, like Steve Keen, like Ann Pettifor, like Joseph Stilitz, like Michael Hudson be heard. Then use democracy to make a choice and give it a generation to make a difference, and then have a review.

Economic Revisionism

Recently the Guardian published an article by J. Bradford Delong (June 29, 2012) Explaining current US Treasury rates is beyond even the economic prophets which tries to rehabilitate mainstream economists. Sadly no comments were possible on the paper's website, but two non-orthodox economists have replied:

Bill Mitchell: Revisionism is rife and ignorance is being elevated to higher levels.

Steve Keen: What utter self-serving drivel, Brad Delong!

Both of these commentators point out in some detail, with extensive quotes and many links, that the people Delong suggests we trust with the economy have always been and still are clueless about what is going on. They are part of the problem not the solution.

Meanwhile the scandal over interest rate fixing is highlighting how corrupt banks and the finance sector have become under deregulation and a culture of greed.