top | item 27467764

Work from home and productivity: evidence from personnel and analytics data [pdf]

86 points| amadeuspagel | 4 years ago |bfi.uchicago.edu | reply

80 comments

order
[+] taneq|4 years ago|reply
> Employees with children living at home increased hours workedmore than those without children at home, and suffered a bigger decline in productivity than thosewithout children.

This tells you all you need to know about the study. You can't analyse the relative effectiveness of working from home and working in the office when you're conflating "remote work" with "trying to get something done on your laptop on the kitchen table while working from home due to the pandemic, while your two early-primary-school-aged kids, who you are simultaneously trying to home-school due to the pandemic, are running amok in the lounge room."

[+] PragmaticPulp|4 years ago|reply
I managed mixed in-office and WFH teams before the pandemic.

There was a stark productivity drop every summer when kids were out on summer break. This includes parents with dedicated home offices who had been WFH for many years.

The effect was so predictable that we planned for lower velocity during summer months. It was also common practice for WFH employees to spend more time in the office for more focus during summer months if they lived close enough.

That’s not to say that kids are the biggest WFH productivity killer. IMO the people who struggled most were the young people who thought they could travel around and work remotely because it doesn’t matter where they’re working from. It turns out it’s plainly obvious that they don’t get much work done when they’re away from home. But that’s another story.

[+] _jal|4 years ago|reply
> This tells you all you need to know about the study.

Your comment only tells me you wanted a different study than the one they did.

Asking what happened in the real world, including in a range of work situations, is a very interesting question to ask.

Looking for/at the delta between optimal work-at-home and the office would also be interesting, for different reasons.

[+] stinkbomb|4 years ago|reply
My company started doing a serious WFH 'experiment' about 18 months before COVID hit. The team I oversee consists of a dozen or so teams of about 40-50 developers each, with a manager that reports directly to me. I don't manage the devs directly, but deal with dev productivity and budgets for (amongst other things) staffing. So our N is not small, but not huge either.

50% of our devs - randomly selected - were offered the chance to WFH, and about 95% of them wound up doing WFH at least 3 days a week, rising to 5 days a week after a few months. The other 50% stayed in the office. The teams that went WFH and the other 50% don't work on the same projects or in the same location.

After about 6 months, we started looking at productivity metrics using a couple Pinpoint-like tools that we built. Simple metrics like: backlog change time, on-time delivery, workload balance, LOC/Checkins ratio, etc. We also tracked things like non-code calendar hours, time logging delays, and other non-development related activity.

Neither group knew what we were tracking or how often. The results were pretty clear as soon as 6 - 9 months in: WFH numbers sucked pretty hard. Everything was down across the board, and it seemed like WFH devs were spending 15-30% more time in non-development tasks, their PR reject rate was as much as 40% higher than office-based devs.

Things were so bad that if COVID had not hit, we were going to abandon the idea of letting people WFH altogether outside of extreme circumstances.

We aren't a unique shop. We're not doing anything out of the ordinary - we're developing on a variety of web and RT device platforms, but doing the same stuff in similar ways as other companies. We have a pretty flat team structure, and teams have a lot of flexibility and decision-making power, which has always worked well for us.

However, it's clear to me and the other execs that WFH absolutely does not work for us.

[+] acituan|4 years ago|reply
I don't buy it.

I mean I trust you got the numbers on whatever metrics you thought you've measured, but I don't buy that this is a well designed human experiment and your conclusions sufficiently exclude alternative explanations.

First systemic bias in your study is that instrumentation of only IC productivity; if IC productivity could exist in isolation, we wouldn't need managerial functions to begin with.

Alternative explanation; distributed work shifts some heavy lifting from the self-organization of ICs to managers actually doing successful multi-party coordination. Most managers suck, therefore they couldn't handle the true test of their function. But having exclusive access to $product-leads@ circle, they could shift blame.

To add more; if productivity was that significantly low for some folks for 6-9 months, and your managers couldn't steer it, that points to a general leadership problem than an IC problem.

Second problem is not having a logical continuity between your IC metrics and business metrics. Did 40% increase in PR rejection rate of half of your developers slowed down any of your business metrics by 20%? E.g release delays, post-release bug counts, decreased reliability etc. If not, then you weren't really measuring what you thought you were measuring.

Alternative explanation; the seat warming, busy work pressure is higher when at the office in the physical presence of peers, which blows up the metrics without meaningful output; WFH makes it much more easy to cut through bullshit work, bullshit meetings etc. Until you can prove that the business output suffered proportionally, metrics like "WFH devs were spending 15-30% more time in non-development tasks" is meaningless. I am not even talking about how hard of a problem it is to really measure "time on dev-task". Sometimes taking a walk in the park to think through a problem is a dev task.

To get a proper picture, you'd need to be able to reject the alternative hypotheses with a greater strength. I think you also need to seriously look into the manager metrics; e.g. how do devs rate them as being able to communicate external priorities and requirements successfully, how many of them think they are working on redundant or meaningless work, that their code complexity is slowing them down etc. Not only these organizational bugs are much more expensive than small-but-easy-to-measure stuff like PR rejection rates, it is also more likely to be in your blindspot because you only interact with managers.

[+] PragmaticPulp|4 years ago|reply
This matches my experience as well: Measurably reduced velocity, noticeably higher rates of rejected PRs and broken builds, and a palpable worsening of productivity and team happiness.

The strangest part is that if you asked individual developers, they would insist that WFH made them more productive and increased their efficiency. Yet managers and the data universally agreed that the opposite was true.

There’s something about WFH that puts distance between individuals and the overall big picture reality in a way that makes these overall issues difficult to see from individual perspectives. It also slowed and dampened all of the feedback loops, giving people the wrong impression that they were working on the right thing or producing proper results until much later. We tried compensating with even more manager effort and over-the-top communication repetition to try to keep everyone informed but it didn’t really help.

There were a few exceptions of people who had identical levels of performance during WFH, but they generally achieved it with significantly more work and proactive communication. Those who struggled the most were juniors who felt like they fell through the cracks.

It’s a difficult topic to discuss on HN because few of us like to admit that WFH is, on average, a struggle for companies. No one likes the idea that WFH comes at a cost.

[+] rickspencer3|4 years ago|reply
> However, it's clear to me and the other execs that WFH absolutely does not work for us.

The "for us" part is refreshing humble for HN comments; thank you for the details of your experience.

I spent about 10 years working on the campus of a very large tech company, and then another 10+ years working for different remote development teams. The remote teams that I have managed were in teams that were built from the ground up to be remote, generally in companies built from the ground up to be remote. And when I say "remote" I really mean "distributed." Hiring the people with the best skills and letting them work from wherever they happen to live rather than relocating them.

I assume in your study that you divided TEAMS into WFM or office based, and did not have teams comprised of some people working remotes and some in the office?

I think that some people are wired to work effectively in distributed teams, some are not. Some managers have the "online social intelligence" to lead distributed teams, some do not. Taking a group of people attracted to working in an office or a high tech campus, and having them "work from home" while the rest of the company is still based in the office does not seem likely to work, though I am surprised by the starkness of your findings.

[+] wcarss|4 years ago|reply
This is really interesting!

Do you know anything about the tools/methodologies of the teams that went part WFH and the adjustments they made when changing up their workstyle?

For example, do you know things like whether meetings were held at their previously scheduled times, vs. whether people started shifting them around to "meet when we're all here", or whether conversations and decisions were pushed organizationally to be more public, visible, and documented, vs made by whoever was in the office, because they could have a hallway chat?

Anecdotally, a friend of mine works at a "remote-first" company, where even when the whole team is in the office, they'll typically meet through zoom/hangouts at their desks, and I have other friends who are "the remote person" on an in-office team. There's a huge gap in the experiences of those two, which is why I ask about those kind of factors in your experience.

[+] mmarq|4 years ago|reply
My company (listed in the LSE) started measuring WFH productivity as soon as Covid started. The unstated objective was to demonstrate WFH reduced productivity, to have an argument to force everybody to go back to the office once the pandemic ended. Metrics were devised by the senior leadership team to serve their purpose. We (dev managers) were tasked with collecting these metrics. After 4-5 months all metrics improved across all teams, so the senior leadership team decided to change them. These new metrics improved as well, so the project was stopped and data was destroyed (lest ICs would find out and have arguments to WFH after the pandemic). Now they’ve started talking about our great office in city center, about the gym, the gaming room, the Friday drinks, the office vibe.
[+] kingdomcome50|4 years ago|reply
Hard truth coming in: The reason it doesn't work for your company is simply poor management (don't take it personally). What you actually measured is the ability of yourself and the rest of your managers to lead your teams in a situation when actual leadership is required.

You see most "managers" aren't really managing anything. "Coordinating" might be a better word. The problem is that coordinating is really just "managing without leadership", and in a WFH environment coordination alone isn't enough. You need people driving your teams, not sitting shotgun with the map reading off directions.

I think the most revealing question is a simple one: What metrics did you come up with to measure the productivity of your managers (and yourself) during this time? None? Of course not... You've probably heard the phrase that "managers are force-multipliers!". Does that apply when the multiplier is less than 1?

I am not disputing your results or even your conclusion, but it's important to know why you got the results you did. And even more important is accurately identifying the appropriate policy going forward: With your current management team you cannot continue with a WFH policy.

[+] osigurdson|4 years ago|reply
It is odd that no one attempted to improve the situation when the metrics started to decline for the WFH group. The only way I could see this happening if the metrics didn't have much correlation with actual results. Presumably this is a real business after all and actual results matter.

Personally I am a little skeptical however since everything sounds so contrived.

[+] tayo42|4 years ago|reply
Does your company help make remote work easier? I guess I could understand that if you took a group used to an office and then forced them to be remote there would be some struggles. I think effective remote work has a bit of a learning curve and needs some investment up front. I think you would need to believe the potential is there and worth chasing.
[+] onlyrealcuzzo|4 years ago|reply
> and it seemed like WFH devs were spending 15-30% more time in non-development tasks

Why did it seem like this and how was it measured?

[+] layoric|4 years ago|reply
Would make a great write up. Interested in the details around what was changed in the company to support this hybrid approach. Also breakdown of devs first time WFH.

From my experience, office->hybrid doesn’t work well as the the staff don’t have the same level of support and processes don’t change to adjust to the change. My first year of WFH (nearly 7 years ago now) was a pretty rough adjustment even though I wanted to do it (glad I stuck with it).

Like most things in life, “Is WFH is more productive?”, “it depends”. Props for doing the experiment and sharing details.

[+] ipaddr|4 years ago|reply
What happened during covid? I assume for sometime you all went remote. Was there a difference in productivity between teams? Did the team with wfh experience lag behind?
[+] foobiekr|4 years ago|reply
Can you describe the kind of work?

This is exactly the experience we are having, especially on new projects.

I’m starting to wonder if this is a high/low tech issue - maybe WFH works great for tasks that are common but not particularly challenging technically. I can tell you point blank that ASIC, systems, etc. work WFH is super broken when people can’t get face to face on a whiteboard or otherwise.

[+] ustamills|4 years ago|reply
The way that your particular study was conducted might be interesting from a study standpoint but it doesn't match the typically accepted ways of doing agile development. For instance if you were studying metrics but not releasing those metrics immediately back to the Developers then there is no opportunity for them to improve from a feedback loop. This seems like a strong invalidation of the study itself.
[+] renewiltord|4 years ago|reply
Interesting data driven approach to the problem. Would you be available to share more details about aggregate outcomes in private? Curious if you were able to slice out common patterns.
[+] geodel|4 years ago|reply
I hope people do not much from this study done in a single IT Service company based in India. One critically important thing to remember is they are "Outlay based" instead of "Outcome based". So things measured are hours billed, time-sheets filled and reports generated. What work got done and was it even half useful to client is of secondary importance at these companies.

I interviewed few years back so thing they ask it if person is married or have kids because they are likely to take more vacations.

I wish they actually detailed the company name. Because if they did and my guess is correct this company is getting hammered as J2EE/.net support model is collapsing. There were some very large layoffs in last few year even before Covid.

[+] reportingsjr|4 years ago|reply
This study matches what I've heard from other companies, for what it's worth.

About a month ago I discussed productivity with someone at one of the large financial companies. They do pretty heavy tracking of employee productivity and noted a pretty significant decrease since everyone's gone WFH.

I don't think it's some other factor that caused this decrease for the company in the study. I'm pretty sure it's consistent across the board.

I voiced this a month or two ago in another thread about WFH, and warned people not to go off of anecdotal productivity claims (which is mostly where I've been seeing the "working from home is significantly better for everything") and got downvoted in to oblivion.

[+] adflux|4 years ago|reply
I have been unfortunate enough to be "diagnosed" with ADHD at a young age. Oh how strange it is that a child can't concentrate with 25 of his peers hopping and chatting around him, let's medicate him as soon as possible! /rant off

Well, a few years ago I rented an office space with a friend of mine, just the two of us in a very spacious (100m2), light and well-ventilated office. We received no phone calls, there were no people walking in, or cats and girlfriends asking for attention. I had less distractions than I had at home, and much less than in a traditional office. And it was a dedicated area for "work". A comfortable desk and chair, two monitors and a fast pc.

All these factors made it so much easier to concentrate and get into the flow... If I had to put a number on it I'd say I would be at least 1.5x more effective working in an area like that compared to a traditional office or at my desk at home.

I will NEVER again say that I am someone who has trouble concentrating, having worked in a place like that.

I wonder if (and how much) the lack of a proper workplace explains the decrease in output when working from home in this study. I can imagine a decrease in productivity if you have to share your home and workplace with other people or even children... But when your children and partner are at work, or if you have a seperate workspace, I wonder how the two compare then.

[+] throwasquirrel|4 years ago|reply
I have been unfortunate enough to not be diagnosed with ADHD until middle age, and I'm sorry to hear about your apparent misdiagnosis but for people who actually do have the condition, it's very real and it very much sucks. It's not just "someone walked into the room and I lost my train of thought", it's "I have one simple task to do and I know exactly how to do it and it's been hours but I still can't make myself do the thing no matter how much I want to."

It's "I haven't done my taxes for 18 months even though it would only take a day, and I cannot force myself to do it no matter how hard I try."

It's "I've been unable to fold my laundry for six weeks even though I've alphabetized my cutlery drawer, twice."

It's "I have to document this project but no matter how much I want to just do it, I nevertheless continually find myself on imgur or Facebook or Hacker News or some random other website reading up about how switched-reluctance motors or Monte Carlo tree search or whatever works."

It's lack of executive function when you need it and it's the inability to think about anything else when something's grabbed your focus and it's a built-in character flaw that you can't "just choose to not have" and it's growing up thinking you're "just lazy" but "have so much potential" and it just. f*king. sucks.

If all you need in order to focus on the thing you want to focus on is for no-one to interrupt you, you don't have ADHD.

/rant

[+] songshuu|4 years ago|reply
While it is given a brief nod in the paper, it is worth reiterating:

WFH during a global pandemic is not the same as WFH generally.

I say this as someone who has been remote for years.

Run this same study starting in September (barring huge Delta variant outbreak in the US) and you'll see a totally different set of numbers.

[+] st8675309|4 years ago|reply
Surprised this post isn’t getting more comments. The result is pretty clear with over 10,000 participants in the study.

>>Total hours worked increased by roughly 30%, including a rise of 18% in working after normal business hours. Average output did not significantly change. Therefore, productivity fell by about 20%. Time spent on coordination activities and meetings increased, but uninterrupted work hours shrank considerably.

[+] geodel|4 years ago|reply
> Surprised this post isn’t getting more comments.

From article:

"The company that provided data is one of the world’s largest IT services companies. They have over 150,000 employees who work with clients across the globe. Most work in the home country, a rapidly- developing Asian nation."

This is about a half-assed Indian "IT Services" company. (From where I am. Need to specify lest I also be branded as racist white dude or whatever). So there is hardly anything to learn in general. Of course similar type IT companies can learn that their lazy micro-managers need to do even more micro managing to get anything done.

The only thing I learned is that one can get even prestigious institution like University of Chicago to do shady shit if there is enough money to motivate.

[+] tetranomiga|4 years ago|reply
>The result is pretty clear with over 10,000 participants in the study.

They're all from the same Asian company, it's far from clear.

[+] carabiner|4 years ago|reply
Interesting. This contradicts the prevailing HN sentiment that WFH increases productivity because there's less time spent socializing or dealing with distractions.
[+] rootusrootus|4 years ago|reply
Anecdotally, I've seen this happen on my team, and it is the single biggest factor that makes me somewhat receptive to going back into the office. It's easier to draw a line between work and home that way.

Maybe we should just turn Slack off altogether outside of business hours, and only enable it in emergencies. LOL

[+] slavapestov|4 years ago|reply
I'm happy to report that my productivity has steadily decreased over the last four years. I credit this to becoming a parent, picking up more hobbies, and working from home. And yet, my compensation has doubled during that time. I really have very little to complain about these days. It's great!
[+] jvanderbot|4 years ago|reply
Vitriolic discussion around this. I'm completely unsurprised. All the key findings are easy to see in my own group. People are working longer hours because they are unaccustomed to working from home near their gaming rigs, or are making up unproductive day hours that their kids interrupted. Its harder to drop in and get quick exchange of information, leading to more meetings, etc.

The danger is that people (c-suite) see this and think it can't be done. It can, but COVID was an imperfect laboratory for studying this.

[+] PragmaticPulp|4 years ago|reply
I managed mixed WFH teams before COVID. It was widely understood that WFH was a productivity obstacle long before quarantines. We usually gave it as a perk to high performing employees who had already proven their ability to get things done and self-manage, or for exceptional employees who couldn’t or wouldn’t relocate near an office.

The idea that WFH is no different than in-office work didn’t really become popular until COVID.

Offices and in-person communication are unmatched for collaboration and communication efficiency. Any company doing work that requires significant collaboration (most tech companies of size) pays a price for WFH. For many companies that overhead is fine if accounted for and managed, but it’s still a price.

[+] pkolaczk|4 years ago|reply
Comparing office work with a WFH situation when kids are at home is unfair - apples to oranges. If you were in the office, where would your kids be? Who would take care of them?

If you want a fair comparison, compare to a situation when you bring your kids to the office. But no one is doing that, right?

WFH works fine if your kids are where they are supposed to be normally: school, daycare, kindergarten, with nanny or with your spouse.

If they can't be taken care of for some reason (e.g. they are ill), you'd be forced to take a leave if you didn't have WFH possibility. So your productivity would be exactly 0. Now with WFH and kids at home maybe you'd not manage to get top productivity, but it would be still greater than 0. So overall it is a net win, even when taking kids into account.

[+] holoduke|4 years ago|reply
Working from home alone is nice the first couple of weeks. Then it becomes boring. Motivation to work decreases significantly. Level of trust to other (mostly new never seen) colleagues become zero. Compassion to the company seizes to exist. Connection to the product, vision and the general 'us' feeling is completely gone. No team spirit, no fun. Yes working from home isn't (in many cases) a big success in the long term.
[+] rijoja|4 years ago|reply
> Connection to the product, vision and the general 'us' feeling is completely gone

Maybe you just realise that the vision and the 'us' feeling is fake.

[+] saas_sam|4 years ago|reply
I remember reading on a different post (can't find it) that productivity went up sharply at the beginning of the pandemic-induced WFH trend, but then subsided over time. The thinking was that people were on their toes early on for fear of being viewed as a slacker, but then once they adjusted to WFH they relaxed. Maybe someone else saw this study...
[+] mikelward|4 years ago|reply
People were running on adrenaline at first. Then the fatigue of having kids at home and not having the usual ways to let off steam eventually hit.
[+] rodcoelho|4 years ago|reply
This doesn't really take into consideration that a lot of businesses/teams within orgs had to quickly pivot strategies due to market conditions. Scrapping plans and shifting priorities last minute, a.k.a. thrash, comes at a cost. So yes it's not totally surprising that people had to work more hours to accomplish a similar output
[+] jaggs|4 years ago|reply
How on earth are you supposed to get meaningful data from a report like this? Does nobody understay the attendant mental health strain of being in the middle of a deadly global pandemic. The fear uncertainty and complete disconnect from family and friends in many cases? And even outside of a pandemic, why should productivity be measured in just work output? Wouldn't it make more sense to look at overall mental health, work/life balance, stress etc? It's widely accepted that workplace stress is increasing, or was before COVID. We're about overdue to try and tackle that surely, before we face a new epidemic of mental health breakdown?
[+] d4nt|4 years ago|reply
Having worked with a few large IT services companies, I’d question their definition of “productivity”. I suspect it’s related to Jira tickets completed or short term revenue (eg billable hours). I doubt it’s measured in terms of end user outcomes.

My experience of these large IT services companies is that their middle management is very focused on maximising profitability for the work their engineers are doing. To the point that they hold engineers back from doing the “right” thing. Eg “don’t add logging to this new feature - it wasn’t explicitly mentioned in the change request”. You could argue this correct from a purist capitalist perspective of course. But that just underlines my belief that outsourcing IT to outside services providers is always more expensive in the long run.

For me, the fact that remote working lessens the influence of middle management is a feature not a bug, even if it hurts the large IT Services provider business model.

[+] sam0x17|4 years ago|reply
Important note: IT != software engineering