How accurate are OECD forecasts?
If you’re a regular visitor to the blog, you’ll know we often report on the latest OECD economic forecasts. After reading these, you may have found yourself asking this question: Is this really what’s going to happen? You’re not alone. The OECD’s economists, too, have been asking themselves the same question, and today they offered some answers.
In a “post-mortem” project, the OECD examined the accuracy of the economic forecasts it issued between 2007 and 2012, a period of seemingly endless economic turbulence. The study reports that, in general, the OECD tended to be too optimistic – it didn’t predict the scale of the collapse in economic activity between 2008-09 and then overestimated the speed of the subsequent recovery.
The errors were particularly notable for certain countries. When it came to the vulnerable Eurozone economies, for instance, the OECD was too hopeful of a quick resolution to the euro crisis. Equally, when it came to small, open economies, the OECD didn’t grasp just how connected these had become to the global economy, making them highly vulnerable to developments beyond their borders.
This review may seem a bit like navel-gazing, but there is a point to it. “We have learned a lot from the crisis,” OECD Chief Economist Pier Carlo Padoan said today in London, where he presented findings from the OECD review. He added that the experience gained over the past few years was helping to change how the organisation works and thinks – a process reflected in the OECD’s ongoing New Approaches to Economic Challenges project.
So what has the OECD learned (pdf)? One quite striking lesson is that the organisation’s economists are better at predicting upturns than downturns (so much for “the gloomy profession”). Between 2007 and 2012, they predicted 91% of accelerations in OECD countries over the following 12 months, but only 46% of decelerations.
The organisations says it also needs to take better account of the impact of the financial system on economies. That’s one reason why it tended to underestimate the impact of weak banking systems on economies. “The repeated deepening of the euro area sovereign debt crisis took us by surprise,” according to Pier Carlo Padoan, “because of the stronger-than-expected feedback between banking and sovereign weaknesses …”.
Of course, this isn’t the first time the crisis has illustrated what many see as a gap in how economists see the world. “There’s a lot of stuff that isn’t there – financial institutions, feedback effects,” William White, Chairman of the OECD’s Economic Development and Review Committee, remarked four years ago in OECD Insights: From Crisis to Recovery. “All of this stuff is very, very hard. I don’t want to disparage current modelling, but the fact of the matter is it’s all very hard. But I do think progress is being made – something has started.”
The review also reflects another oft-repeated theme from the crisis – namely, the tendency towards “groupthink,” or a herd mentality, in economic and financial analysis. As we reported here on the blog some time ago, an IMF self-evaluation in 2011 attributed some of its failures in the run-up to the crisis to “a high degree of groupthink” and “an institutional culture that discourages contrarian views”. Indeed, it’s notable the extent to which OECD forecasts between 2007 and 2012 matched those of other leading international agencies like the IMF, World Bank and European Commission. A lesson for the OECD, perhaps, is that it needs to encourage its people to “think out of the box” (apologies for the jargon).
But perhaps the most important lesson to come out of the crisis is that governments, economists, international organisations need to come to terms with the reality that they know less than they think they do. That might sound like a platitude, but it has important implications for economic forecasting, some of which are already evident in OECD work. Most notably, the OECD is now placing more emphasis on the health warnings surrounding its projections; in other words, it’s saying “here’s what we think will happen, but here’s why it might not”. Worth bearing in mind next time you read a forecast on the blog.
OECD Forecasts During and After the Financial Crisis: A Post Mortem (pdf) (An OECD Economics Department Policy Note)
New Approaches to Economic Challenges at the OECD