top | item 39332834

Undisclosed tinkering in Excel behind economics paper

194 points| _Microft | 2 years ago |retractionwatch.com

138 comments

order
[+] bjornsing|2 years ago|reply
A related anecdote: Jönköping University is not allowed to call itself a university in Swedish (“universitet”), since that word is, by law, reserved for institutions that meet certain requirements. To get around this it uses the English name “Jönköping University” in Swedish as well. But formally it’s a so called “högskola”, more often translated as ”college”.
[+] pelorat|2 years ago|reply
To be honest the difference is minimal. I studied CS at Mitthögskolan in Östersund, but didn't complete all points before graduation (still had a paper to write and hand in for one of the courses). A year after graduation, Mitthögskolan became a real university (Mittuniversitetet). I emailed the professor in charge of the course I had not yet completed, wrote my paper and handed it in, then received a university diploma in computer science, even though i technically never studied at a "real" university.
[+] fullspectrumdev|2 years ago|reply
Until recently in Ireland, third level education places were largely divided into “universities” and “institutes of technology” (previously “regional technical colleges”).

The institutes of technology have been reorganised lately and are now called “technical universities”.

The degree you get is generally much the same from any of them - but for some insane reason graduates of the institutes of technology/technical universities don’t get to vote in the national elections for the upper house (senate). Only graduates of the more prestigious ones get that right.

[+] em500|2 years ago|reply
Similar as the Netherlands: there 37 "hogescholen" marketing themselves as "University of Applied Sciences" in English that are not allowed to claim to be "universiteiten" in Dutch (a name is reserved for 14 more academically advanced institutions).
[+] lancebeet|2 years ago|reply
This is a really unfortunate but also quite common pattern in Sweden, where there's a veneer of internationality which tricks foreigners instead of helping them. Swedes will generally understand that Jönköping university isn't formally a university and wouldn't compare it to the more prestigious schools, but for international students it's not at all obvious. They might pay a lot of money and discover too late that the education and/or credentials they get in return don't match at all what they paid for. As usual, the Swedish authorities seem unwilling or impotent to take action on this type of issue.
[+] jeltz|2 years ago|reply
I wonder if that would really stand up in court. Here in Sweden our judges are not very impressed by clever technicalities.
[+] 7734128|2 years ago|reply
Chalmers on the other hand is technically a universitet, but maintains the högskola title in Swedish.
[+] zdimension|2 years ago|reply
In France, "Licence" and "Master" (Bachelor's and Master's degree respectively) are protected diplomas that can only be delivered by universities (or by schools that depend on universities).

To circumvent this, private non-state-certified schools deliver things called "Bachelor", "MSc", "MBA" or "Mastère" that walk and quack like regular diplomas but since they're using those specific names, they're not actually state-recognized diplomas. Which means that if a company is hiring "people with a Licence" they can reasonably deny someone with a "Bachelor" because even though it's also a 3-year degree, there is no guarantee whatsoever that it's actually worth anything. Anybody can create a company and hand out sheets of paper with "MSc in Psychoceramics" written on it.

This is a big problem for high schoolers who usually don't know the difference and end up paying thousands of euros for private schools that deliver close-to-worthless degrees (not all are worthless, but their worth is mostly correlated to how well-known the school is, whereas if you get a real state-certified degree from a random university anywhere in France it'll be recognized by any company).

[+] apt-get|2 years ago|reply
Similar thing in France, however related to degrees: a Master's Degree fits within the common framework of European tertiary studies, AKA your 2-year graduate cycle that ends up totalling 5 years if you include a standard undergrad cycle (Licence). However, a "mastère spécialisé" (which is an actual trademark) is not an officially recognized diploma at all, but a label used by schools to denote (usually paid and expensive) "specialization paths". They provide no academic credits, and you can't move forward to a PhD with one. Many prestigious schools (grandes écoles) provide those kind of certifications to clueless foreign students who end up paying huge sums of money for something that holds no formal weight in any academic framework.
[+] wodenokoto|2 years ago|reply
Wouldn’t that also make the professor _not_ a professor?
[+] oohffyvfg|2 years ago|reply
in most countries the term university is reserved for institutions which provide graduate (undergrad optional) degrees in four (three in some places) major fields (sciences, humanities, medicine, etc)

nothing else.

usually just adding a business major to a place with medicine, engineering and law is enough to cross the threshold and jump from college to uni.

[+] nsajko|2 years ago|reply
For comparison, would Harvard or Stanford be allowed to call themselves an universitet?
[+] thimp|2 years ago|reply
Oh this is nothing. Go look at the social sciences. They get data that compromises their entire profession, tell the university it’s inconclusive and wait until the research grant runs out and fail to publish. Then do the same thing again.

I have seen this with my own eyes. A bit of excel drama is usually behind their published papers. Also seen that with my own eyes.

The field has trouble collecting data too because it’s expensive and very subjective. The worst one I saw was somehow turning 7 data points via imputation and interpolation into a hundred or so. The principal researcher on that I actually had to teach BIDMAS to and her calculations were completely whacked out (excel again)

[+] screenoridesaga|2 years ago|reply
A lot of people are upset when their grant doesn’t get funded and it’s easy to suppose there’s some sort of sinister motive behind it. In truth it’s probably that the science is very difficult and only a tiny fraction of people get anywhere in social sciences.

Try marine biology, they are always looking for decent papers.

[+] FrustratedMonky|2 years ago|reply
Because there are bad apples, doesn't mean you should throw out the scientific method.

Over 50% of social science studies do replicate.

Sorry that isn't high enough. We should definitely just give up on understanding ourselves.

Or, maybe fix the financial incentives.

Any industry across all fields, if the incentives are miss-matched, will produce crap.

[+] concordDance|2 years ago|reply
What profession have you seen compromised?
[+] dash2|2 years ago|reply
"Journal of Cleaner Production" is not "highly ranked" in economics. Economists are pretty hierarchical about their journals and know which are the top 5, which are top field journals etc. That one is a non-entity, I have literally never heard of it.
[+] boppo1|2 years ago|reply
What are the top 5?
[+] grey-area|2 years ago|reply
The title is a bit odd. The article details a sloppy use of substitute data from adjacent rows or even columns to fill gaps in data, excel has little to do with it. The worst example is using United Kingdom data to fill gaps in USA data because they were adjacent in the list.
[+] smt88|2 years ago|reply
Excel has everything to do with it. From the article:

> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data.

The most charitable interpretation is that the professor has no idea what autofill is or isn't capable of doing, so he misused it.

[+] riedel|2 years ago|reply
Such things are encouraged by the excel UI because it fails to make relationships and manual corrections easily visible. The human computer interface matters beyond UX.
[+] scotty79|2 years ago|reply
Also Netherlands to New Zealand. Hilarious.
[+] croes|2 years ago|reply
Excel makes such things convenient
[+] readthenotes1|2 years ago|reply
This reminds me of an error in the Excel sheet used by the EU to justify austerity programs back in 2008 the led to trouble in the PIGS countries.

Someone double checked that and found that there was an error in the equation is correction show that austerity programs were mildly worse for recovering from economic disasters.

--

On an editorial note, it seems to me that economics is just highfalutin astrology and this does nothing to convince me otherwise

[+] immibis|2 years ago|reply
Many excuses have been used to justify austerity. If this study didn't work, they'd use another. If 99 studies said austerity was bad and 1 study said it was good, I suspect that one would be more likely to get published; in either case it would get used to justify austerity. It seems to me like austerity is the pre-determined conclusion, and then they go looking for reasons.
[+] raid2000|2 years ago|reply
I expected to read a paper about some obscure Excel trick to manipulate stats output. Instead, this is just old-fashioned manipulation by hand or "imputation" as the paper describes it.

> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data. He had marked anywhere from two to four observations before or after the missing values and dragged the selected cells down or up, depending on the case. The program then filled in the blanks. If the new numbers turned negative, Heshmati replaced them with the last positive value Excel had spit out.

[+] MostlyStable|2 years ago|reply
The crazy thing about it is that the author doesn't seem to understand why it's bad. He doesn't appear to be hiding it. He just says "yeah that's what I did, whoops, I forgot to say it in the paper". He's either decided that acting like a complete moron is better than being thought of as an an intentional fraud, or else he really does think it was totally above board.
[+] FrustratedMonky|2 years ago|reply
""Is the evidence for austerity based on an Excel spreadsheet error?"

Lets not forget that people/politicians also will ignore errors if they support their point of view. This can lead to policy decisions that impact the global economy.

The US 'Right' believe strongly in biblical/moral individual responsibility, so any study that supports that gets a pass.

This lead to 'austerity' measures after 2008. Because "everyone should just be more responsible in their spending".

""They actually found, using a different method, that the economic growth would be around 2.2 percent. Reinhart and Rogoff admitted that their spreadsheet was accidentally omitting 5 rows in an Average formula, and claimed that the corrected result would be a positive growth but of just 0.2 percent.""

https://www.powerusersoftwares.com/post/2016/08/11/the-excel....

https://www.theguardian.com/politics/2013/apr/18/uncovered-e...

https://www.washingtonpost.com/news/wonk/wp/2013/04/16/is-th...

[+] lkdfjlkdfjlg|2 years ago|reply
You post on HN, people see it as a UI problem. Everything is tech, everything is tooling.

But if you knew anything the history of science, you'd know that a sloppy scientist with good tools gets you sloppy science, and a diligent scientist with bad tools might still get you as-good-as-possible science.

[+] LeifCarrotson|2 years ago|reply
People aren't uniform, but they're all human.

Improvements in tooling raise the baseline universally, making the "sloppy" scientists malfeasance obvious and helping the "diligent" scientist avoid inadvertent mistakes.

Moralizing about humans being humans and wishing we were better gets us nowhere, but technology carries us into the future.

[+] SebFender|2 years ago|reply
“The reason it’s cheating isn’t that he’s done it, but that he hasn’t written it down,” Sorry but no. Every day my team takes decisions that have significant risk management impacts and we clearly state the subjective implications. Even when we have numbers to back things up, we always state the nature of our decisions.
[+] bouncing|2 years ago|reply
> If I had other intensions [sic] and did not believe in my imputation approach, I would not share the data with him.

Admittedly this isn’t my field. But that sounds a lot like “I only show my work to people who will validate it,” does it not?

[+] timmb|2 years ago|reply
I disagree, I think it means “my actions have demonstrated that I’m not trying to cover anything up”
[+] scotty79|2 years ago|reply
I'm not surprised that results of this study were not surprising.
[+] moron4hire|2 years ago|reply
This is called "econometrics".
[+] nequo|2 years ago|reply
This is not econometrics though. Actual econometrics is upfront that you have missing data, performs partial identification rather than pretending that you have point identification, and gives you upper and lower bounds on what the true parameter value might be.

Only if you did that with this paper, you’d have bounds so wide that no useful conclusion could be drawn, as indicated by the author’s statement that without imputation such data is useless.

[+] 745frm|2 years ago|reply

[deleted]

[+] blackbear_|2 years ago|reply
Did you really just make an account to post this comment? That's cowardly, and sad.