top | item 35758301

(no title)

jameslevy | 2 years ago

It's amazing to me how often Goodhart's Law can be observed in the wild, and I've come to believe it is an important enough concept that it should probably be something (along with logical fallacies and other concepts related to critical thinking) that should be prioritized above a lot of other things taught in high schools.

discuss

order

novok|2 years ago

So when you tell execs goodhart's law, they ask, ok what can we do about it when managing large organizations? Be it a 200 person department of a 30'000 person organization or even more?

onos|2 years ago

The usual argument is you need a metric and a check metric that can catch gaming the first. Eg unit sales and mean price per sale — the latter preventing a race to the bottom on price point.

bruce511|2 years ago

The best approach is really hard. (Which is why proxy metrics in the first place.)

The best approach is to have a -very- clear set of goals, and to accurately measure progress to those goals, not to proxy metrics or goals.

Yeah, I know, easier said than done.

Usually a business has multiple goals, which are complementary. Enumerating, and measuring these make the system harder to game. Or more accurately make it harder to get better metrics without also getting closer to the actual goals.

If you want sales,then measure sales. If you want growth measure that. If you want customer satisfaction then measure that. If you want happy employees then measure that. List -everything-, measure everything. Understand the tensions, improvement in one area can drop the metrics in another. So work to keep improving them all.

All of this is -hard-. So most places don't bother. Find a simple measure (say stock price or turnover or profit) and ignore sustainability etc.

BurningFrog|2 years ago

If Goodhart's Law becomes commonly known, I expect some next-level fallacy will emerge in its place.