top | item 5559281

Researchers Finally Replicated Reinhart-Rogoff, and There Are Serious Problems

367 points| rdp | 13 years ago |nextnewdeal.net | reply

282 comments

order
[+] cs702|13 years ago|reply
Stepping back for a moment, this thorough debunking of Reinhart and Rogoff's flawed paper means that now there is NO evidence that government debt exceeding 90% of GDP somehow negatively impacts growth.

Let me repeat that: there is NO EVIDENCE that more government debt causes slower economic growth -- just theories.

I wonder what all the government deficit scaremongers will say to this!

--

PS. For reference, the often-cited paper by Reinhart and Rogoff can be downloaded at http://www.nber.org/papers/w15639.pdf and the new paper debunking it can be downloaded at http://www.peri.umass.edu/fileadmin/pdf/working_papers/worki...

--

PS#2. An Excel typo was partly to blame for Reinhart and Rogoff's errors! Reminds me of the Excel typo that allowed the "London Whale" silently to rack up billions of dollars in unexpected losses at JP Morgan: http://baselinescenario.com/2013/02/09/the-importance-of-exc...

[+] JumpCrisscross|13 years ago|reply
"there is NO evidence that government debt exceeding 90% of GDP somehow negatively impacts growth."

Correct, 90% is not the magical, universal debt/GDP:dGDP inflection point it was made out to be.

"there is NO EVIDENCE that more government debt causes slower economic growth"

Whoah, whoah. Try the refutation paper [1]. It finds "that average GDP growth tails off somewhat when the public debt/GDP ratio increases towards 120 percent."

There is no strong inflection point ("the scatterplot does suggest a non-linearity in the relationship, but that occurs in the change in the public debt/GDP ratio from 0 to 30 percent"). But there is a monotonically decreasing debt/GDP:dGDP relationship.

It is more subtle than previously characterised, and thus potentially mitigatable in the short term, but let's not take Einstein's refutation of Newtonian physics to mean gravity doesn't exist.

[1] http://www.peri.umass.edu/fileadmin/pdf/working_papers/worki...

[+] fauigerzigerk|13 years ago|reply
There may not be empirical evidence for the magical 90% level, but we don't need empirical evidence to know that at some point interest payments will require highter tax revenues which will reduce disposable income and hence consumption.

Obviously someone is on the receiving end of those repayments, but if that someone is not putting money in the pockets of those same taxpayers and instead decides to finance a power plant in Thailand then GDP will inevitably suffer.

I'm not a government deficit scaremonger by any means. I think Krugman's (and others') argument that the U.S. government can borrow very cheaply and should therefore act in a countercyclical manner for the common good is plausible. I'm also convinced that the crowding out argument is bogus in an economy that suffers from too little demand whilst corporations are sitting on tons of cash.

But we should not lose sight of the fact that interest rates are a highly psychological affair. Once trust in the ability to repay debt gets eroded things can get out of control very quickly. So we should be scared at least a little bit.

That said, the U.S is probably the one entity on this planet that is furthest away from that point. Even Japan with it's 245% debt/GDP ratio can borrow at 0.5% for 10 years whilst their currency is crashing and the central bank is announcing a massive QE program twice the size of the U.S one.

But if the U.S ever gets to the point where Japan is now, that's when I'm starting read up on agriculture or the hunter-gatherer diet.

[+] haberman|13 years ago|reply
It seems to me that "deficit scaremongers" are more worried about scenarios like Greece and the Weimar Republic, not "slower economic growth."

I think it's a good-faith concern. As much as I believe in data, the ability of the US to borrow cheaply enough to finance deficit spending is entirely up to the whims of investors. People are fickle. If the shit hits the fan and investors lose confidence in the USA for whatever reason, the deficit spending we currently rely on could become many times more expensive, and unsustainable.

EDIT: Downvotes? I doubt most people are even aware of this 90% number, but everyone knows about Greece. If you ask an average tea partier, I am willing to bet they are more concerned about Greece than exceeding 90% debt/GDP. If this is nonsense, refute it (I'd like to hear, and believe, that refutation).

[+] grannyg00se|13 years ago|reply
How can huge government debt not cause slower economic growth? At very least all of the money wasted on debt repayment could've been spent on incentive programs. Unless the borrowed money is invested in something that consistently returns more than the debt financing, I don't see how one could think a high debt/gdp ratio would be okay.
[+] bubbleRefuge|13 years ago|reply
There never was any evidence because they never published the data till now. Nobody could reproduce the results.
[+] protomyth|13 years ago|reply
I would imagine the problem cited will be the amount of revenue going to debt service.
[+] bcoates|13 years ago|reply
Is there evidence to the contrary or are we back to people yelling past each other with unsupported theories?
[+] kaonashi|13 years ago|reply
Lost in the conversation about government debt is the determining factor of whether or not government debt can ever be a problem at all. It's not how much you owe, it's not who you owe it to; it's what you owe.

If your government owes IOU's that it can print itself (fiat currency), it can never default on those IOU's.

If your government owes IOU's printed by another government or sovereign body, then it can most certainly run out of them and must buy them at market rates.

[+] tunesmith|13 years ago|reply
So, yay Keynesianism?
[+] anigbrowl|13 years ago|reply
Stepping back for a moment, this thorough debunking of Reinhart and Rogoff's flawed paper means that now there is NO evidence that government debt exceeding 90% of GDP somehow negatively impacts growth. Let me repeat that: there is NO EVIDENCE that more government debt causes slower economic growth -- just theories.

These two sentences do not say the same thing. There is evidence that debt results in slower economic growth. There is not evidence that it negatively impacts growth, to with that it results in economic shrinkage. I'm with your general sentiment but don't throw the baby out with the bathwater in your enthusiasm.

[+] WalterBright|13 years ago|reply
I'm not so sure that this error means that a free lunch has been discovered.
[+] ianstallings|13 years ago|reply
What us "scaremongers" are going to say is that it's not very wise. Debt is based on future growth so it's always a gamble. Wouldn't you rather be prudent and spend less? Or is there some rush to spend every single dime? I just don't get it. Breaking it down with research into past events doesn't negate the fact that borrowing has to be paid back. Did we just fall of the rational-thinking cart or something? It's not silly to be conservative. In fact in a lot of places it's called wisdom.
[+] oleganza|13 years ago|reply
Since the GDP contains all government spendings and debt, it is obvious that debt and spendings only increase "economic growth". But if you call a spade a spade and realize that 100% of government spendings is taken from products of the market, then the real GDP (RP) will be defined like this: RP = "Official GDP" - 2 * (gov debt and expenditures).
[+] duaneb|13 years ago|reply
I guess I just assumed that if excel was used at a financial firm, there were unit tests (or whatever the excel equivalent would be). It's a scary thought that it might not be the case because excel doesn't excel at reporting errors or debugging.
[+] Vivtek|13 years ago|reply
They'll say nothing at all.
[+] bernardom|13 years ago|reply
Is The Reinhart-Rogoff Result Based On A Simple Spreadsheet Error?

Oh my goodness, no. That's the least of it. They took all Commonwealth countries, found the periods where they had over 90% debt-to-GDP ratio, and EQUAL WEIGHTED them regardless of length or country.

I won't do a better job of explaining this than the article: "U.K. has 19 years (1946-1964) above 90 percent debt-to-GDP with an average 2.4 percent growth rate. New Zealand has one year in their sample above 90 percent debt-to-GDP with a growth rate of -7.6. These two numbers, 2.4 and -7.6 percent, are given equal weight in the final calculation, as they average the countries equally."

Wow.

[+] cletus|13 years ago|reply
This actually isn't news. Questions have already been raised as this paper was never peer reviewed [1] (Jan 2013), which states:

> According to their C.V.s [2], it's been published in the May 2010 issue of the American Economic Review, which is a special non-reviewed "papers and proceedings" issue.

So I really don't know why anyone listened to this in the first place other than it furthered their pre-existing political agenda.

This is why any paper should be required to release:

- the raw data supporting it;

- the assumptions, exclusions and weightings made to produce the final data; and

- any code used for a model.

All of these issues are particularly problematic when it comes to climate science.

[1]: http://www.nextnewdeal.net/rortybomb/no-90-percent-debt-thre...

[2]: http://www.carmenreinhart.com/c.v./cv-in-pdf/

[+] lbarrow|13 years ago|reply
Wait -- so apparently it's acceptable in economics to publish big important papers with broad-reaching public policy implications without releasing the data on which the paper was based? That would never fly in the sciences.
[+] sigil|13 years ago|reply
> That would never fly in the sciences.

Or would it?

http://climateaudit.files.wordpress.com/2009/12/mcintyre-grl...

Regardless of what side you come down on w.r.t. this particular issue (and I realize it's a sensitive one), the fact that global policy was influenced by a paper that used secret, buggy Fortran code to manipulate a data set is extremely concerning. This is not good science.

Publishing data and source code should be a requirement these days.

[+] icelancer|13 years ago|reply
Yes, though this has been a huge charge for guys like Levitt/Dubner (Freakonomics authors) - that data sets must be published in tandem with conclusions based on said data. (Assuming, of course, the data sets are not publicly available. It does no good to reproduce census data and the like!)

You should see how bad it is in my field of study - biomechanics. It's a pure black box. We have total cronyism, where one reviewer has disproportional power since he was the first to publish much of the modern stuff. Therefore, people will do anything to have him review their papers, and his papers are pushed through despite what many feel are seriously glaring errors in methodology and collection.

This is the walled garden crap that turns me off to "peer-reviewed research."

[+] james_ash|13 years ago|reply
Feynman talked about an "extra" kind of integrity, where one bends over backwards to show how one might be wrong. In my field (organic chemistry), this is de rigeur, and it is enforced by the fact that you can release a paper on one day and have people report on how replicable it is within 24h.

For a much fuzzier field like economics, where the line between knowledge and opinion is extremely blurry, this "bending over backwards" to show how you might be wrong should be applied 100x .

[+] jules|13 years ago|reply
Umm, that's not true. In plenty of fields it's common not to publish the raw data, and instead publish only some graphs that summarize a huge data set down to a couple of data points. This includes biology, physics, and medicine. On the contrary, people actively "protect" and "guard" their data. And yes, that's a big problem.
[+] pc86|13 years ago|reply
Welcome to the social sciences.

This is in part why I am a programmer now and why my Poli Sci degree is sitting in a drawer.

[+] protomyth|13 years ago|reply
If that offends you, wait until you get a social science paper with conclusions that are contradicted by the data sets included in the paper.
[+] dbecker|13 years ago|reply
That would never fly in the sciences.

It's pretty common in medical research.

[+] snowwrestler|13 years ago|reply
Yes it would because the test of scientific knowledge is whether it be confirmed with a different data set.

If a conclusion is only provable with one specific data set, then it can hardly be considered a universal objective truth of nature.

By analogy: if I drop a hammer and tell you that gravitational acceleration is 9.8 meters per second squared, you don't need to come over to my house and borrow my hammer to test it.

Sharing data and code is helpful for error checking, as it was in this case.

[+] _delirium|13 years ago|reply
In this case the main publication that had impact was a book, which even in the physical sciences tend to be reviewed under different standards. There is some pretty out-there stuff published by major physicists in book form.
[+] martythemaniak|13 years ago|reply
The most incredible and depressing aspect of this is that it will have no impact whatsoever on policy makers, leaders or their supporters.

Can anyone imagine Paul Ryan or Cameron/Osborne coming out and admitting their policies are based on nothing more than gut feelings unsupported by anything? What about the countless pundits who've spent years squawking about austerity? The notion is laughable.

[+] trotsky|13 years ago|reply
It's interesting to note that one of the authors of the original paper, Ken Rogan, was the chief economist at the IMF 2001-2003. He also joined the group of 30 in 2008, an economic policy organization that is very influential in things like the Basel accords, IMF governance and matters of finance in Brussels.

Many of these organizations rely on private data to support their policy decisions, or rely on private analysis from similar organizations. That's hard to criticize because at least some of it would cause serious harm to the folks who provided it, or enable high finance players to front run policy actions to huge profit.

But what to make of an insider like that refusing to release data that had no market risk or legal restrictions in place. It's not hard to imagine that his primary goal was support of an economic philosophy and would have been just as happy to publish a paper that claimed high debt/gdp rations promote growth.

It certainly makes me wonder if this kind of approach is part of the culture in some of these policy and international finance organizations. If the data relied on by the IMF/ECB/etc as they've effectively reformed governments and imposed major budget changes can't be made public, then you're highly reliant on them to be ethical and extremely diligent.

If there's any significant amount of philosophy trumping science in the european restructuring, one begins to wonder if some of the weaker euro members are still actual democracies.

[+] Uhhrrr|13 years ago|reply
I see a lot of comments here to the effect that this means debt is a free lunch now. Would that it were so - we could get rid of taxes, as well as Medicare and Social Security withholding. And nix the Obamacare penalty. And, of course, buy everyone a pony.

Sadly, ponies are a finite resource until such time as the government figures out how to print more of them. Debt still has a multiplier less than 1 - far less, for large amounts of debt.

All this article means is the magic number where debt starts to cause deep hurting is somewhere above 90% of GDP.

[+] TallGuyShort|13 years ago|reply
Serious question - not trolling. Even if increasing the deficit does not affect GDP (and I'm in no position to even comment on whether or not I think it does) - how is that sustainable? I mean, AFAIK the argument behind increasing government spending is as simple as "it solves problems" - it provides help to those in need, affects the economy, etc. But why won't this at some point collapse? And if it will at some point collapse, why do we think politics will allow the government to behave any differently as we near that time? So again - serious question - I fail to see how we can expect government spending to grow without eventually consuming 100% of everyone's income or collapsing entirely (and I understand that many people may consider the former a good thing).

edit: And this may be because I don't know where the money comes from to begin with. If I keep getting bigger and bigger loans and there's no way my income will allow me to pay these loands back in the foreseeable future, then won't everyone just cease doing business with me at some point? Does "debt" in a government context act differently, and if so, why?

[+] bloaf|13 years ago|reply
Some historical notes:

http://en.wikipedia.org/wiki/File:USDebt.png

The US national debt during WWII was around 120% of GDP. In the years that followed, that ratio fell to below 40%. However, we didn't appreciably pay off our debts, the real dollar amount stayed a little over $2 trillion. We accomplished this feat, of course, by substantially increasing the GDP. It is easy to realize that it is fine for debt to continuously grow, so long as the GDP grows faster.

Also, the only president to entirely pay off the national debt, Andrew Jackson, triggered one of the deepest depressions in US history.

[+] derefr|13 years ago|reply
> I fail to see how we can expect government spending to grow without eventually consuming 100% of everyone's income or collapsing entirely (and I understand that many people may consider the former a good thing).

Both options sound good to me, in that if people manage to keep eating and living in houses and raising their children after this happens, we will have officially entered a post-scarcity economy.

[+] crapshoot101|13 years ago|reply
This surprises me - Reinhart and Rogoff are both very well respected economists, and neither is a culture warrior type - they've published enough (Including "This Time Its Different", a great book to understand the history of financial crises) that I almost wonder if there's more to this than meets the eye at first glance. I realize the first tendency is to "burn the witch", but I'd like to learn more.
[+] 3am|13 years ago|reply
"both very well respected economists"

Not any more. This is a career killer. I wouldn't be surprised to see them accepting posts with the American Enterprise Institute or Cato in short order.

EDIT: also, Barry Ritholtz is generally perceived as an honest broker on these things, and his commentary is here: http://www.ritholtz.com/blog/2013/04/did-reinhart-rogoff-scr...

[+] johngalt|13 years ago|reply
Debt and growth are obviously separate entities that are hard to correlate. Intuitively it makes sense that debt doesn't inherently slow growth, and could even speed growth.

Reducing the problem to a personal level, you borrow money to buy a good car which enables you to work a higher paying job. This scenario makes the case that debt causes growth. From a policy standpoint if you borrow money for long term infrastructure and growth opportunities you could easily see debt as a net positive.

OTOH, you run up your credit cards to buy a better TVs or to buy 'friends'. In this instance debt can only lead to eventual poverty and ruin.

Which type of spending do you think is happening more often at a federal level?

[+] jl6|13 years ago|reply
Isn't it more plausible that public debt is caused by government deficit spending, which causes economic growth, but a temporary kind of growth that is only as good as the government's ability to continue spending?
[+] ArkyBeagle|13 years ago|reply
Dropping back the frame for a bit - Jeff Sachs is on EconTalk this week, and he says what ( IMO ) should be the idea under scrutiny - employment prospects for people without college degrees have declined quite a bit since 1973. The present relatively high debt/deficit load was a result of an attempt to trade debt/deficits for higher employment using a ... somewhat intentionally-blown housing bubble. It didn't work. Something mysterious is "regulating employment down" in the system, and we don't seem to have a good handle on it. Tyler Cowen has posited a weakish "zero marginal product worker" hypothesis that does not seem to be easily decidable. Sachs' idea is that this has been developing since about 1973 and we haven't addressed it yet, and that this failure is a failure grounded in weaknesses of social science itself.

I can't find a more evenhanded and clear statement of the problem than his.

[+] danso|13 years ago|reply
> In a new paper, "Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff," Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst successfully replicate the results. After trying to replicate the Reinhart-Rogoff results and failing, they reached out to Reinhart and Rogoff and they were willing to share their data spreadhseet. This allowed Herndon et al. to see how how Reinhart and Rogoff's data was constructed.

What's amazing to me is that no one, until the researchers spotlighted here, apparently thought to ask to see the data? Isn't this part of peer-review?

Things like this is why I love being part of the open-source community. People may brag about and diss each other's code performance to an unnecessary degree of hostility, but at least we can all check the code and run it for ourselves.

[+] jdrobins2000|13 years ago|reply
Putting on my tinfoil hat, I wonder if this error was intentional or unintentional? Was the errant conclusion known to be false by any who propagated it? If so, by whom and to what end?

For example, perhaps the US and Euro governments wanted to justify some tough measures of fiscal restraint and increase public support, and they wanted to keep it from being delayed until collapse was impossible to avoid. (See, I'm not totally jaded. But I can think of plenty of worse scenarios too.)

The "flaws" seem to be too large (and some deliberate) to be unintentional, but maybe I give too much credit. I've just noticed too much misinformation being spread by those who should know better, and coincidentally seem to benefit the most from it. I'm suspending my judgement until I learn more, but anyone else have any insights?

[+] narrator|13 years ago|reply
Mainstream economics is centered around recommending policies that ensure bond-holders are paid. It all leads back to that. If there is too much debt destroying the underlying economy causing bond interest payers to default, lower rates. If rates go to zero, print money to pay the bondholders. If that causes inflation raise taxes and cut spending to pay the bondholders (a.k.a austerity), etc. The political economic dimension of the relationship between bondholders and the people who pay there interest is an important missing component from our understanding.
[+] seoguru|13 years ago|reply
Randy Wray's view: http://www.economonitor.com/lrwray/2013/04/17/no-rogoff-and-...

concluding paragraph makes so much sense:

"More than five decades ago, Abba Lerner gave the answer to this question. If there are involuntarily unemployed (we would add underemployed) people it means the deficit is too low. The government should either cut taxes or increase spending. It is certainly debatable which one is a better policy, but that’s beyond the scope of this paper. When is the deficit too large? When it’s over 3%, 7%, 10%? Again, there is no magic number and anyone who comes up with a universal number simply misunderstands the modern monetary regime and macroeconomics. In opposition to magic, Lerner proposed “functional finance”—the notion that the federal government’s budgetary outcome is of no consequence by itself, but rather, what is important is the economic effects of government spending and taxing. When total spending in the economy, including government spending, is more than what the economy is able to produce when employed at full capacity, the government should either lower its spending or raise taxes. A failure to do so will lead to inflation. So inflation is the true limit to government spending not lack of financing. Government debt is merely the result of government deficit and hence the same applies to debt as well."

[+] kybernetikos|13 years ago|reply
Are we describing running the same formulae on the same numbers as 'replicating' these days?

Did real economists always consider these results provisional/dubious/bogus and the fact that they were used as the basis of policy is to the shame of our policy makers and press, or is the field of economics so messed up that research based on schoolboy errors can become the accepted view within the field?

[+] api|13 years ago|reply
It seems clear to me that government debt in modern fiat-credit economies isn't really debt in the same sense that private debt is debt. It doesn't behave like private debt. I don't think debt is even the correct term for it. Maybe we need a new one.