Now having worked in a few smaller teams since my time at FB, I realize how marvellous it is to not have to worry about OKRs, performance reviews and all that malarkey. It was so draining to run yourself as a mini PR/brand within one's team (/one's department) and market oneself (one's team) to your manager (/your company).
So much mental bandwidth was spent.
I also, from a systems-thinking POV, just fundamentally reject the value in OKRs as a meaningful proxy for the value you (/your team) are producing unless you are an entirely mechanistic function. By mechanistic I mean you have a clear discrete Input<>Output expectation. Every single time an element of human activity is reduced to a metric, you lose something, and if you do it enough times, you've effectively produced a metric that is completely distinct from the thing you intended on measuring in the first place. For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted'. Well, that sounds good, but unbeknownst to you, perverse incentives have kicked in and invisibly imbued your metric-chasing experiments a dark undertone. A new risk model might start flagging more content as potentially suicidal to boost numbers, leading to over-intervention that could traumatize users or waste resources. Or perhaps the surface we use to measure success, a UX or user activity metric, might actually be uncorrelated with crisis aversion or... and this is where it hits: the thing that's instrumental in crises is social media itself. The metric becomes a game, and it games _you_. Your team thinks you're doing X, but you're really doing Y. Because you're disattached from the real problem and comforted by incredibly lossy proxies.
The times I've seen OKRs and metrics used effectively, they met two criteria. (1) it was always a metric closely aligned with what customers want and which drives revenue for the company; and (2) it didn't just impose a requirement to do work, it also provided political cover for not doing other, competing work.
For example, if you're designing GPUs and/or GPU drivers? If your next-generation product has the aim of providing 25% more frames per second in "Baldur's Gate 3" and "Call of Duty 6" while maintaining the same quality - that would be a good objective for the team, as it's closely aligned with what your customers want.
And if someone should come to you and tell you there's a lot of people streaming these days, and they think you should optimise gaming FPS and h264 compression at the same time? It's a sensible request but it's also a distraction; proper goal setting will let you say "great idea, but not this quarter".
But there are a lot of fields of endeavour where it's not possible to meet these criteria - like the suicide prevention team from your example.
I'm experiencing large company culture for the first time after being in small startups. What I object to the most, is the competition between engineers created from the goal setting and performance reviews. At startups, I had a domain that I could own. Now, at largco, everyone tries to take over my area in an effort to build their resumes. I used to view fellow engineers as teammates, now I see them as my competition.
I have semi-humorously dropped a comment defining Goodhart's law.
The problem you are describing is nothing else but Goodhart's law in action: A measure stops being a good measure, i.e. be a proxy for something, once there are objectives attached to it. In other words, attaching goals to a metric invalidates previous causal relationship.
That's neither bad, not good. It's a property of goal setting. The problematic part is still treating the measure as if it had causal relationship to something when that relationship has already been invalidated. In your example, number of suicides ceases to be comparable between pre and post OKR timeframes, however if you look closely, this particular goal is based on a metric the underlying OKR targets invalidate.
Yes, sometimes you get these weird tautologies where you have to change the whole framework/process to make something both targetable and measurable simultaneously, potentially losing comparability to past data.
As a team within a larger company, your purpose is to contribute to the larger goals. How do you know if you are doing that? As this "suicide prevention team", how do you know if you are doing a good job?
I agree with you that proxy metrics easily distract you into doing Y instead of X. My opinion is that you need to iterate on your metrics then. Not having metrics means it all depends on the gut feeling of executives.
It surely is draining to be clear about your goals. I fear we cannot really be politically correct and sufficiently honest even. What is the real goal of having a suicide prevention team? It might be token effort after some incident, then the actual goal would be to as cheap as possible while still maintaining the illusion. It might be to prevent future PR disasters, then collection helpful evidence for lawyers should be part of the job. This touches hard ethical questions and these should become evident when discussing the purpose of a team.
You have invoked Goodhart's Law. The problem is, of course, that most managers are not good at their task of evaluating talent, proving the worth of their services, etc. and try to take the easiest way out of it. Sometimes this means outsourcing the job to you or picking a poor thing to measure.
> It was so draining to run yourself as a mini PR/brand within one's team (/one's department) and market oneself (one's team) to your manager (/your company).
Counter point. This is always inevitably a thing. They were only making the implicit explicit.
Safari’s idiotic AI has decided that every time I want to go to youtube, I actually want to go to a specific clip of peppa pig failing to learn how to whistle
Google’s idiotic AI has decided that the clip of peppa pig failing to whistle depicts suicidal ideation, and so every time I visit the clip I get a big dramatic black rectangle and a therapyspeak question “Am I an adult who is prepared to view a depiction of suicide.” It took me embarassingly long to notice that this nessage, repeated constantly enough, was actually affecting my mood.
Your scenario is generous in assuming that the suicide prevention FAANGers who coded up this situation are intelligently following bad incentives. I think its more likely that their intelligence is just found lacking, when unfairly compared to the galaxy brain needed to actually guess the consequences of our actions at this scale.
> For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted'
That sounds like the sort of old-school KPIs that OKRs were meant to replace. I don't know if it's just impossible to measure anything, and you should just rely on a managers' word for how any team is doing, or if the people who did KPIs are now infecting OKRs.
The challenge is we don’t have an alternative, at a small company your performance boils down to “does the ceo want to fire you?”. The extent to which the ceo does not want to fire you depends on the reasonableness of that CEO, as well as how much the CEO cares.
In an established small/medium business with flat growth, it’s entirely possible that no one cares to fire anyone, sits also possible the CEO expects everyone to work nights and weekends while being a top competitive coder.
One of the biggest culture shocks I had during my time at FB was when we were doing Mononoke, this completely greenfield project with a ton of unknowns (first big Rust project!), and we got a new skip level who was previously on the web performance team (super narrow and directed).
When I was at fb, I had the curse of being given an extremely vague goal as a new hire. Unfortunately, it was quite difficult to establish meaningful metrics AND hit them in a half! Like you said, I spent way more time on my self review than I wanted...
I remember reading an anecdote about this stuff from the book The Essential Deming by W. Edwards Deming the "father of the quality movement and was hugely influential in post-WWII Japan".
It talked about some company moving oil around in barges, the setup was unprofitable and a new manager was brought up to fix it. He created metrics of the business, but didn't put any goals or goalposts for the barge crews and other managers in the company.
All he did was simply print the metrics and glue them on every barge of the fleet weekly. Soon the whole project was massively profitable. Apparently the captains of the barges would compete with each other to see who could save more money, without any penalties for the low performing nor any rewards for the high performing barges.
One thing that was very surprising is that the barges were given freedom to choose where to source materials and, more importantly, the fuel. Before the metrics the captains would pick whatever was closest no matter the prices, after the metrics the captains would, on their own, try to source fuel from the cheapest place that they could. No amount of central planning could have been as optimal as the prices fluctuated a lot from day to day, so whenever a captain saw a good deal he would re-fuel.
It turns out that empowered people want to do a good job and want to save money.
The barge company in question is Koch Industries (yes, that Koch Industries). Starting in 1982 they leveraged Deming’s ideas, realized 30% profits, and pushed it far enough to enter the dark side.
> Koch’s dedication to Deming’s ideas eventually led the company into several sticky situations, not the least being targeted in a Senate Select Committee investigation for oil theft in 1988, a direct result of immense internal pressure on employees as part of its continuous improvement program.
[1]
This is powerful stuff. When you empower people and set a goal, they will do anything it takes to hit that goal, including breaking the law.
> It turns out that empowered people want to do a good job and want to save money.
You have to be careful here. In your example those captains didn’t really care about their job they cared about getting the high score. Metrics, KPIs, etc work but only when they’re setup perfectly aligned with your actual business goals. Measure/score the wrong thing and you’ll get nowhere.
For example, what if the metric posted was distance traveled by a barge. You’d have captains taking the longest routes possible just to get the high score instead of shipping the most product per unit time which is what the manager would have intended.
I think business people understand the value of setting high level goals and giving autonomy to accomplish them.
It’s a good sign if your job works this way. Unfortunately this tends to mostly be applied at the VP level. Engineers are modeled as expensive pieces of an equipment to optimize and derisk.
This works when people can compete on something measurable. But when you have teams split into feed, messaging, and ads, something else might be needed to measure value.
I've often speculated about a radical interpretation of this idea, inspired by a old video game (whose name I forget). In the game, you rule a kingdom, but unlike, say, Civilization, you don't directly manage things. You set goals and create quests, like "slay this monster and get some reward." And the quests would inspire heroes to join your kingdom, and things would grow from there. Iirc, you create incentives around your economy as well.
Imagine if there were product goals ("implement feature X") with some reward [1] attached and you could leave it up to teams or individuals to claim that goal if they desired. You could choose the goals you wanted to claim, recruit coworkers to help you, (eg, self form teams). PMs/Management would basically be in charge of allocating rewards for the goals.
I imagine it'd be a terrible system in practice for a number of reasons, but I enjoy thinking about ways you could attempt to make it workable. For example,
[1] rewards -- I don't think you could tie rewards directly to people's paychecks. Do that too much and I think you'd create perverse incentives. But perhaps things like swag, gifts, time off, or just bragging rights, honor, and glory might work.
[2] coordination -- a danger would people redundantly working on the same goal. You'd need a way to prevent that.
[3] other perverse incentives -- you might get an overabundance of folks choosing the "fun" goals, for example. (After all, engineers may be more motivated by that than other things.) Here I imagine the rewards for unsexy things would need to rise over time if nobody opted for them. Or, you make first dibs on some other "fun" goal the prize for achieving a less fun goal.
>It turns out that empowered people want to do a good job and want to save money.
This has more to do with Japanese culture than anything. Do that in America, and you just get signs with 'Over 99 Billion Served'
It becomes meaningless to everyone.
The reality of OKRs (or Rockefeller Habits or any other company wide framework for establishing priorities and goals,) is that by going through the exercise of trying to create them across the company, you often discover that there are meaningful disconnects in how teams are prioritizing work or how teams function vs how other stakeholders in the company believe those teams are operating. The real value comes from diving deeper into understanding those things. However, companies often view those as annoyances and find a way to plow the goals through while completely missing that learning opportunity.
This. High level OKRs should be set by the company to steer the ship, and engineering OKRs should align with the company OKRs. Thus, the roadmap is defined by the OKRs - I believe that's the whole point.
Large organizations also just have to be more process-driven and formal. And it's fine if you don't like that. At a former employer, I was much happier when it was a medium-sized enterprise than when it grew by almost 10x. A lot of things had to change that weren't really to my liking (or skill set) but they had to.
I've always worked in a non-tech where our customers are internal.
They have unfortunately read too many SRE/Phoenix Project/Big Tech Management Papers and so OKRs was one thing shoved down our throats.
What I've found is that they've hired a slew of Product Mangers (tm) who mostly just sit in between devs & users, without coordinating cross team, actually writing specs/Jiras/roadmaps, etc. So each teams devs wrote their own OKRs, in ignorance of what users (who might be other tech teams) might actually want or what other teams OKRs are.
For example, to use an analogy one team builds the foundation and decides their OKR is to use 15% less concrete next year. The other team builds frames and decides their OKR is to make the average building 15% taller next year. If any of them talked to actual final customers, they'd have found out the customers actually desire the buildings to be more robust to storm winds.
Am I the only one who doesn’t care about OKRs or whatever companies are following nowadays? I tend to work around 2-3 years per company. Get a salary bump when switching. My first year is rather slow/smooth because “I’ve been here for less than a year!”. My second year is more interesting and I do take it as “real” work. My third year I couldn’t care less since I’m already looking for something else.
I couldn’t care less about goals or OKRs. I get paid, I solve whatever problems the business has (whether they make sense or not) and then I just leave.
I always like to observe people who want quick results reading about agile or OKRs or KPIs and then handsomely shooting themselves into foot.
Other fun thing is having management mandate those things and people who don’t have time to understand all that stuff implement it.
Worst one is having ambitious young people thinking if they push for implementing those things they will be „real professionals” - that is how dev teams go to crawl enforcing all BS rules and product teams bikeshedding OKRs instead of delivering stuff.
I’ve led teams and then I’ve worked as an engineer, so I feel I understand both sides but have an informed take on how things go when OKRs show up despite/due to that.
I get as a leader that some form of goal orientation and niche engineering speak translation into outcomes must occur.
But, for every 1 leader who can do this well, there are 99 who can’t. For every 1/2 leaders who can do this faithfully, there are 99.5 who don’t care much beyond building fiefdoms or just plain don’t know how to do it in practice - the ex-consultant PM, the manager not the leader, the PM who holds zero authority over a team of skilled ICs, and so on.
Also, as an engineer, I have OKRs, I have ops, and I have the random stuff that shows up that blows a hole in my week to week plans. A good PM maybe can reduce this, but again see above. And: what am I measured on for my hearth and home paychecks: Answer - OKRs.
So, in practice, when OKRs show up, I believe earnest big picture effort (which people love going to startups for, I think), goes out the window overall because of the above.
You’ll only get people hitting OKRs, “hitting OKRs” has so much absurd flex in it because again see above, and so you best hope you have the 1 of 100 PMs that know how to do that well or else people start doing the silly dances engineers leave companies over.
I’ve had the same experience, both as IC and a manager. It’s a nice framework, just not for human organizations (KPI for next framework: result in less “no true Scotsman” replies whenever a post details why the framework sucks in their organization)
In my experience managemant roles set the OKRs in a closed circle, without feedback from technical employees and then things like reliable infrastructure are underestimated or forgotten. Once OKRs are set, they are set in stone, because management roles are too busy to come together again and additionally include engineering roles in the decision making process. Instead just put more pressure on engineering.
This kind of thing easily destroys great team dynamic and makes people change from getting work done to meeting target metrics, even if they only game them. This can lead to good and essential people quitting and the downfall of the company.
I strongly believe that the addition of every metric brings with it an associated productivity tax in the form of:
1. time spent doing things that exploit this metric
2. time spent purely documenting/surfacing this metric on your ‘brand’
This post reflects an insidious anti pattern in the practice of setting OKRs: "shipping the roadmap" is not the objective, it is a means to achieving some other underlying objective.
With a well written objective / key result (ex: "grow DAU by 30%"), you can abandon your entire roadmap two weeks into the quarter and still hit your OKRs. They enable you to respond to new information and lessons learned, rather than locking the whole team into a rigid plan for the entire quarter.
OKRs and KPIs are scams. Human language and psychology are both vague and malleable. The kinds of people who are good at meeting OKRs aren't the kinds of people who can produce quality work. OKRs are mostly about lowering expectations prior to implementing and exaggerating results after implementing... It's mostly psychological work, a magic trick, not real productivity.
The goal of this game is to sacrifice long term gains for short term gains; build a house of cards fast to boost your OKR scores and get promoted fast... Then let the next person who is assigned to the project take the blame for the collapse.
People don't care about foundations... Only thing that matters is whose watch the tower falls on. People are literally that ignorant and superficial. It works every time.
I'm convinced there is no 'good' way to set goals/OKRs etc. in big companies. Orgs should set goals though, but the problem is when you spend too much time doing it.
If top level management set company OKRs, and then cascade them down with every department and sub department then setting and aligning mini OKRs to the big ones you end up with a months long planning cycle which by the time you finish, you need to restart for the next year.
A good heuristic is that no team should ever end up in a situation where they spend more time debating how a task should be prioritized than the amount of time the task takes to do.
As an engineer I have nothing against OKRs. But it's often the day-to-day management (EM or PM) who tend to override the team's OKRs for upper management to prioritize something ad-hoc & out of scope they agreeably said Yes to.
This article is interesting, because we as a team are currently going through a strange phase, which is similar to KPIs and OKRs, but in a better way than previous companies did it.
Some internal discussions and skewed perceptions of our team are now causing our team are currently causing our team to make our work and our infrastructure more tractable and more quantifiable. For example, we've started to track time spent for internal customers and different topics team-internally and aggregate those on the team-level. Or we've started to generate information about resource consumption of different products.
And this has led to interesting results. At a workers level, our director and the board is very clear that they want to continue this data to be collected without interference or skew. If you work in an area, you track it in that area - don't try to protect anyone. Track time as you spend it. Track resources as spent.
On a leadership level, this has however resulted in a fairly interesting tone. Suddenly you have a CEO saying "Alright, so my very expensive team is spending that much time on this area you claim to be somewhat simple? Make that number go down significantly this year. Hire if and as necessary". Or "I dislike this amount of outages, but if you say we need better data there, make it so"
It's also a very much data driven management style. But you don't have magical bespoke numbers fall from the sky one day and you need to make sense of them and integrate them into your normal work. It is data about our work we collect along the way, and we're trying to change our tasks and our decision making to change our work to change the metrics into a better direction.
Goodhart's law suggests that no fixed set of measures will suffice, but what about a dynamic or a set of complementary measures that periodically switch? Spend 3-6 months optimizing for X, then spend 3-6 months optimizing for some property complementary to X. Like having metrics that focus on new features, then bug fixes, then new features, etc. Or new features, bug fixes, refactoring/maintenance, new features, bug fixes, etc.
Selecting the set of good and complementary metrics requires careful thought and experimentation, but it might prevent gaming over the long term because the complementary measures of the second phase should make up for deficiencies in the primary measures in the first phase. Anyone have any experience with this sort of thing?
This is another way to determine if a company is a "start-up"! If you work at a company that has OKRs, it's no longer a start-up or it's time to bail because said start-up is about to fail.
The problem is that OKRs are divorced from the “go do this piece of work” instruction, and so, just like code comments, they quickly become out of date or are outright ignored.
Because of that, the prescription about SMART and focus on delivery is simply wasted effort. At best a duplication.
The good part about OKRs is (/should be) that it forces an alignment conversation between teams and amongst managers. And the performance discussion with manager is often useful and sometimes revealing. More often both focus on metrics, losing the last opportunity to extract value from the concept.
[+] [-] padolsey|1 year ago|reply
So much mental bandwidth was spent.
I also, from a systems-thinking POV, just fundamentally reject the value in OKRs as a meaningful proxy for the value you (/your team) are producing unless you are an entirely mechanistic function. By mechanistic I mean you have a clear discrete Input<>Output expectation. Every single time an element of human activity is reduced to a metric, you lose something, and if you do it enough times, you've effectively produced a metric that is completely distinct from the thing you intended on measuring in the first place. For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted'. Well, that sounds good, but unbeknownst to you, perverse incentives have kicked in and invisibly imbued your metric-chasing experiments a dark undertone. A new risk model might start flagging more content as potentially suicidal to boost numbers, leading to over-intervention that could traumatize users or waste resources. Or perhaps the surface we use to measure success, a UX or user activity metric, might actually be uncorrelated with crisis aversion or... and this is where it hits: the thing that's instrumental in crises is social media itself. The metric becomes a game, and it games _you_. Your team thinks you're doing X, but you're really doing Y. Because you're disattached from the real problem and comforted by incredibly lossy proxies.
[+] [-] michaelt|1 year ago|reply
For example, if you're designing GPUs and/or GPU drivers? If your next-generation product has the aim of providing 25% more frames per second in "Baldur's Gate 3" and "Call of Duty 6" while maintaining the same quality - that would be a good objective for the team, as it's closely aligned with what your customers want.
And if someone should come to you and tell you there's a lot of people streaming these days, and they think you should optimise gaming FPS and h264 compression at the same time? It's a sensible request but it's also a distraction; proper goal setting will let you say "great idea, but not this quarter".
But there are a lot of fields of endeavour where it's not possible to meet these criteria - like the suicide prevention team from your example.
[+] [-] eweise|1 year ago|reply
[+] [-] friendzis|1 year ago|reply
The problem you are describing is nothing else but Goodhart's law in action: A measure stops being a good measure, i.e. be a proxy for something, once there are objectives attached to it. In other words, attaching goals to a metric invalidates previous causal relationship.
That's neither bad, not good. It's a property of goal setting. The problematic part is still treating the measure as if it had causal relationship to something when that relationship has already been invalidated. In your example, number of suicides ceases to be comparable between pre and post OKR timeframes, however if you look closely, this particular goal is based on a metric the underlying OKR targets invalidate.
Yes, sometimes you get these weird tautologies where you have to change the whole framework/process to make something both targetable and measurable simultaneously, potentially losing comparability to past data.
[+] [-] qznc|1 year ago|reply
I agree with you that proxy metrics easily distract you into doing Y instead of X. My opinion is that you need to iterate on your metrics then. Not having metrics means it all depends on the gut feeling of executives.
It surely is draining to be clear about your goals. I fear we cannot really be politically correct and sufficiently honest even. What is the real goal of having a suicide prevention team? It might be token effort after some incident, then the actual goal would be to as cheap as possible while still maintaining the illusion. It might be to prevent future PR disasters, then collection helpful evidence for lawyers should be part of the job. This touches hard ethical questions and these should become evident when discussing the purpose of a team.
[+] [-] dfxm12|1 year ago|reply
[+] [-] loloquwowndueo|1 year ago|reply
Basically Goodhart’s law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
[+] [-] jhghikvhu|1 year ago|reply
Counter point. This is always inevitably a thing. They were only making the implicit explicit.
[+] [-] QuadmasterXLII|1 year ago|reply
Google’s idiotic AI has decided that the clip of peppa pig failing to whistle depicts suicidal ideation, and so every time I visit the clip I get a big dramatic black rectangle and a therapyspeak question “Am I an adult who is prepared to view a depiction of suicide.” It took me embarassingly long to notice that this nessage, repeated constantly enough, was actually affecting my mood.
Your scenario is generous in assuming that the suicide prevention FAANGers who coded up this situation are intelligently following bad incentives. I think its more likely that their intelligence is just found lacking, when unfairly compared to the galaxy brain needed to actually guess the consequences of our actions at this scale.
[+] [-] robertlagrant|1 year ago|reply
That sounds like the sort of old-school KPIs that OKRs were meant to replace. I don't know if it's just impossible to measure anything, and you should just rely on a managers' word for how any team is doing, or if the people who did KPIs are now infecting OKRs.
[+] [-] lumost|1 year ago|reply
In an established small/medium business with flat growth, it’s entirely possible that no one cares to fire anyone, sits also possible the CEO expects everyone to work nights and weekends while being a top competitive coder.
[+] [-] sunshowers|1 year ago|reply
[+] [-] roland35|1 year ago|reply
[+] [-] theonething|1 year ago|reply
Where are these malarkey-free companies? I've worked at several small companies and they all had all kinds of malarkey.
[+] [-] theGnuMe|1 year ago|reply
We measure something because we need something to measure even if it is divorced from reality.
[+] [-] epolanski|1 year ago|reply
Also, metrics eventually become the goal and are gamified.
[+] [-] TheBigSalad|1 year ago|reply
[+] [-] treetalker|1 year ago|reply
[+] [-] DanielHB|1 year ago|reply
It talked about some company moving oil around in barges, the setup was unprofitable and a new manager was brought up to fix it. He created metrics of the business, but didn't put any goals or goalposts for the barge crews and other managers in the company.
All he did was simply print the metrics and glue them on every barge of the fleet weekly. Soon the whole project was massively profitable. Apparently the captains of the barges would compete with each other to see who could save more money, without any penalties for the low performing nor any rewards for the high performing barges.
One thing that was very surprising is that the barges were given freedom to choose where to source materials and, more importantly, the fuel. Before the metrics the captains would pick whatever was closest no matter the prices, after the metrics the captains would, on their own, try to source fuel from the cheapest place that they could. No amount of central planning could have been as optimal as the prices fluctuated a lot from day to day, so whenever a captain saw a good deal he would re-fuel.
It turns out that empowered people want to do a good job and want to save money.
[+] [-] vegetablepotpie|1 year ago|reply
> Koch’s dedication to Deming’s ideas eventually led the company into several sticky situations, not the least being targeted in a Senate Select Committee investigation for oil theft in 1988, a direct result of immense internal pressure on employees as part of its continuous improvement program.
[1]
This is powerful stuff. When you empower people and set a goal, they will do anything it takes to hit that goal, including breaking the law.
[1] https://commoncog.com/deming-paradox-operational-rigour/
[+] [-] chasd00|1 year ago|reply
You have to be careful here. In your example those captains didn’t really care about their job they cared about getting the high score. Metrics, KPIs, etc work but only when they’re setup perfectly aligned with your actual business goals. Measure/score the wrong thing and you’ll get nowhere.
For example, what if the metric posted was distance traveled by a barge. You’d have captains taking the longest routes possible just to get the high score instead of shipping the most product per unit time which is what the manager would have intended.
[+] [-] liontwist|1 year ago|reply
It’s a good sign if your job works this way. Unfortunately this tends to mostly be applied at the VP level. Engineers are modeled as expensive pieces of an equipment to optimize and derisk.
[+] [-] 2OEH8eoCRo0|1 year ago|reply
At a factory it's easy to know how many widgets were made. It's tricky with software.
[+] [-] muzani|1 year ago|reply
[+] [-] everybodyknows|1 year ago|reply
Sounds good, but how did the "metrics" expose which individual decision-makers were responsible for any improvement or deterioration?
[+] [-] msluyter|1 year ago|reply
Imagine if there were product goals ("implement feature X") with some reward [1] attached and you could leave it up to teams or individuals to claim that goal if they desired. You could choose the goals you wanted to claim, recruit coworkers to help you, (eg, self form teams). PMs/Management would basically be in charge of allocating rewards for the goals.
I imagine it'd be a terrible system in practice for a number of reasons, but I enjoy thinking about ways you could attempt to make it workable. For example,
[1] rewards -- I don't think you could tie rewards directly to people's paychecks. Do that too much and I think you'd create perverse incentives. But perhaps things like swag, gifts, time off, or just bragging rights, honor, and glory might work.
[2] coordination -- a danger would people redundantly working on the same goal. You'd need a way to prevent that.
[3] other perverse incentives -- you might get an overabundance of folks choosing the "fun" goals, for example. (After all, engineers may be more motivated by that than other things.) Here I imagine the rewards for unsexy things would need to rise over time if nobody opted for them. Or, you make first dibs on some other "fun" goal the prize for achieving a less fun goal.
[+] [-] 1970-01-01|1 year ago|reply
This has more to do with Japanese culture than anything. Do that in America, and you just get signs with 'Over 99 Billion Served' It becomes meaningless to everyone.
[+] [-] SteveVeilStream|1 year ago|reply
[+] [-] AndrewSChapman|1 year ago|reply
[+] [-] ghaff|1 year ago|reply
[+] [-] zoul|1 year ago|reply
[+] [-] rfdearborn|1 year ago|reply
[+] [-] doppp|1 year ago|reply
[1] https://x.com/benwbear/status/1543056694330003456
[+] [-] steveBK123|1 year ago|reply
They have unfortunately read too many SRE/Phoenix Project/Big Tech Management Papers and so OKRs was one thing shoved down our throats.
What I've found is that they've hired a slew of Product Mangers (tm) who mostly just sit in between devs & users, without coordinating cross team, actually writing specs/Jiras/roadmaps, etc. So each teams devs wrote their own OKRs, in ignorance of what users (who might be other tech teams) might actually want or what other teams OKRs are.
For example, to use an analogy one team builds the foundation and decides their OKR is to use 15% less concrete next year. The other team builds frames and decides their OKR is to make the average building 15% taller next year. If any of them talked to actual final customers, they'd have found out the customers actually desire the buildings to be more robust to storm winds.
[+] [-] dakiol|1 year ago|reply
I couldn’t care less about goals or OKRs. I get paid, I solve whatever problems the business has (whether they make sense or not) and then I just leave.
[+] [-] ozim|1 year ago|reply
Other fun thing is having management mandate those things and people who don’t have time to understand all that stuff implement it.
Worst one is having ambitious young people thinking if they push for implementing those things they will be „real professionals” - that is how dev teams go to crawl enforcing all BS rules and product teams bikeshedding OKRs instead of delivering stuff.
[+] [-] dogman144|1 year ago|reply
I get as a leader that some form of goal orientation and niche engineering speak translation into outcomes must occur.
But, for every 1 leader who can do this well, there are 99 who can’t. For every 1/2 leaders who can do this faithfully, there are 99.5 who don’t care much beyond building fiefdoms or just plain don’t know how to do it in practice - the ex-consultant PM, the manager not the leader, the PM who holds zero authority over a team of skilled ICs, and so on.
Also, as an engineer, I have OKRs, I have ops, and I have the random stuff that shows up that blows a hole in my week to week plans. A good PM maybe can reduce this, but again see above. And: what am I measured on for my hearth and home paychecks: Answer - OKRs.
So, in practice, when OKRs show up, I believe earnest big picture effort (which people love going to startups for, I think), goes out the window overall because of the above.
You’ll only get people hitting OKRs, “hitting OKRs” has so much absurd flex in it because again see above, and so you best hope you have the 1 of 100 PMs that know how to do that well or else people start doing the silly dances engineers leave companies over.
[+] [-] maayank|1 year ago|reply
[+] [-] zelphirkalt|1 year ago|reply
This kind of thing easily destroys great team dynamic and makes people change from getting work done to meeting target metrics, even if they only game them. This can lead to good and essential people quitting and the downfall of the company.
[+] [-] justinl33|1 year ago|reply
[+] [-] friendzis|1 year ago|reply
[+] [-] jtbetz22|1 year ago|reply
With a well written objective / key result (ex: "grow DAU by 30%"), you can abandon your entire roadmap two weeks into the quarter and still hit your OKRs. They enable you to respond to new information and lessons learned, rather than locking the whole team into a rigid plan for the entire quarter.
[+] [-] jongjong|1 year ago|reply
The goal of this game is to sacrifice long term gains for short term gains; build a house of cards fast to boost your OKR scores and get promoted fast... Then let the next person who is assigned to the project take the blame for the collapse.
People don't care about foundations... Only thing that matters is whose watch the tower falls on. People are literally that ignorant and superficial. It works every time.
[+] [-] crowcroft|1 year ago|reply
If top level management set company OKRs, and then cascade them down with every department and sub department then setting and aligning mini OKRs to the big ones you end up with a months long planning cycle which by the time you finish, you need to restart for the next year.
A good heuristic is that no team should ever end up in a situation where they spend more time debating how a task should be prioritized than the amount of time the task takes to do.
[+] [-] qprofyeh|1 year ago|reply
[+] [-] keeptrying|1 year ago|reply
- the perception of hitting OKRs is more important than anything else
- if you can enter into the OKR discussion early, then you can control it.
- if you can, run the OKR process much to your gain :)
[+] [-] tetha|1 year ago|reply
Some internal discussions and skewed perceptions of our team are now causing our team are currently causing our team to make our work and our infrastructure more tractable and more quantifiable. For example, we've started to track time spent for internal customers and different topics team-internally and aggregate those on the team-level. Or we've started to generate information about resource consumption of different products.
And this has led to interesting results. At a workers level, our director and the board is very clear that they want to continue this data to be collected without interference or skew. If you work in an area, you track it in that area - don't try to protect anyone. Track time as you spend it. Track resources as spent.
On a leadership level, this has however resulted in a fairly interesting tone. Suddenly you have a CEO saying "Alright, so my very expensive team is spending that much time on this area you claim to be somewhat simple? Make that number go down significantly this year. Hire if and as necessary". Or "I dislike this amount of outages, but if you say we need better data there, make it so"
It's also a very much data driven management style. But you don't have magical bespoke numbers fall from the sky one day and you need to make sense of them and integrate them into your normal work. It is data about our work we collect along the way, and we're trying to change our tasks and our decision making to change our work to change the metrics into a better direction.
[+] [-] naasking|1 year ago|reply
Selecting the set of good and complementary metrics requires careful thought and experimentation, but it might prevent gaming over the long term because the complementary measures of the second phase should make up for deficiencies in the primary measures in the first phase. Anyone have any experience with this sort of thing?
[+] [-] shusson|1 year ago|reply
[+] [-] dent9876543|1 year ago|reply
Because of that, the prescription about SMART and focus on delivery is simply wasted effort. At best a duplication.
The good part about OKRs is (/should be) that it forces an alignment conversation between teams and amongst managers. And the performance discussion with manager is often useful and sometimes revealing. More often both focus on metrics, losing the last opportunity to extract value from the concept.