Big (and small) companies routinely make "data-driven decisions" based on obviously-faulty data collection, bad models, or misapplication of statistics. It rarely takes some crack scientific mind to spot it, either. But they want a decision now and doing it right might take a lot longer or cost more, or the correct model might be a lot fuzzier and that makes them uncomfortable, or they've got some notion in their heads already and they'll be damned if mere numbers are going to get in the way of that, so everyone in the room's just supposed to nod along when the blatantly-biased graphs come up on the Power Point suggesting (erroneously) that we do X.
The business world runs at least as much on bullshit as the most cynical among us might think, I'd say. It's not half as clever or competently-run as one might hope, certainly.
It's a cliche that front-line workers have a better understanding of customers and products than the c-suite and that this leads to predictable blunders, because that's often true.
(I'm also not sure I'd call Reddit a "giant corp", but that's beside the point [EDIT] and anyway, to be fair, this particular discussion isn't just concerned with Reddit)
Reddit is far from giant in terms of employees or revenue, and you'd be surprised at how many dumb decisions and how much money is wasted by startups that have raised hundreds of millions of dollars. You really shouldn't assume competence in situations like these, especially when Reddit has recently been making a bunch of terrible long term decisions to try and juice their numbers in the short term for IPO
I think the problem is that people read the analyses that companies share publicly (e.g., in a press release or earnings call), and they assume that that's all they did.
You should assume that any numbers shared publicly are just the tip of a giant-ass iceberg and that there were probably 10x more analyses going on internally that weren't shared.
The thing that ultimately gets shared publicly is whatever avoids using advanced stats or internal jargon; They want a single soundbite, not a scientific paper with a full methods section.
Story time: Worked at a large multinational, and 6-7 years ago they decided they had to ape into the whole “data science” gold rush. They spent millions of dollars on hardware, software, salaries, and consulting.
After a year with not much to show for it, the VP for the silo starts to put out kudos for the team for break-even revenue impact. However, behind the scenes, the insight they were taking credit for was a common sense idea that had already been in the e-commerce team’s backlog.
Nothing surprises me when a company says that they’ve run the numbers.
Me too. Reddit almost certainly ran those numbers.
They might have chosen to ignore them. They might have fed biased assumptions into their calculations. Project management might have messed up and deprioritized the features meant to placate that section of the userbase. Institutional churn might have resulted in the corporation forgetting that they ever did run those numbers or why they are now heading down this path.
Having worked at some quite large companies: yes, it happens for real that they do not do stuff like that. It is likely that some team obviously has ran the numbers but that does not mean they made it to the people who made the decision.
People really think a giant corp acts a monolithic entity with perfect distribution of knowledge and ability and the stupid actions of dumb individuals will always be stamped out rather than amplified.
bamfly|2 years ago
The business world runs at least as much on bullshit as the most cynical among us might think, I'd say. It's not half as clever or competently-run as one might hope, certainly.
It's a cliche that front-line workers have a better understanding of customers and products than the c-suite and that this leads to predictable blunders, because that's often true.
(I'm also not sure I'd call Reddit a "giant corp", but that's beside the point [EDIT] and anyway, to be fair, this particular discussion isn't just concerned with Reddit)
xorcist|2 years ago
ren_engineer|2 years ago
parpfish|2 years ago
You should assume that any numbers shared publicly are just the tip of a giant-ass iceberg and that there were probably 10x more analyses going on internally that weren't shared.
The thing that ultimately gets shared publicly is whatever avoids using advanced stats or internal jargon; They want a single soundbite, not a scientific paper with a full methods section.
coredog64|2 years ago
After a year with not much to show for it, the VP for the silo starts to put out kudos for the team for break-even revenue impact. However, behind the scenes, the insight they were taking credit for was a common sense idea that had already been in the e-commerce team’s backlog.
Nothing surprises me when a company says that they’ve run the numbers.
phire|2 years ago
They might have chosen to ignore them. They might have fed biased assumptions into their calculations. Project management might have messed up and deprioritized the features meant to placate that section of the userbase. Institutional churn might have resulted in the corporation forgetting that they ever did run those numbers or why they are now heading down this path.
jeltz|2 years ago
relaxing|2 years ago
And yes comments like this amaze me too.