[lotharbot]

> "the view that only the objective is real ... has been increasingly producing absurd results."

As you rightly noted a couple of comments back, what this view does is it smuggles in subjective assumptions. That is, someone operating under this view is going to objectively measure something (like kloc or number of tickets closed or execution time on a test data set) but the selection of what to measure, and the selection of how to value each individual measure of an objective quantity in order to determine overall "goodness", is subjective. The step where they assign meaning to a measurement is a subjective step.

It's interesting to watch the development of "objective" measures in basketball and the dialogue around how to determine if a player is the best, most valuable, etc. over time. Decades ago, the only stats we had were "counting stats" -- points, rebounds, and assists. Steals and blocks came a bit later. There is a correlation between putting up big counting stats and winning games, but it's not as strong as you might naively suppose. Once more sophisticated metrics were developed, something that "subjective" observers had always noticed ("losing player with good stats" is something that was often said about specific players) started to be quantified: some players put up big stats because they're doing inefficient things that result in individual stats at the expense of the team, like taking a high volume of shots even if they're lower percentage shots than a teammate could get on that play, or not contesting an opponent's shot but trying to chase the rebound instead (leading to more opponent scoring but also more personal rebounds over the course of a large number of shots.) In the modern era, advanced stats like PER, VoRP, WS, and BPM are basically more sophisticated models built on top of counting stats that try to scale them and weight them according to regression models. These stats are better, but they still don't capture everything, they only capture things that can be inferred from counting stats. They don't capture things like -- Steph Curry has such strong "shooting gravity" that his teammates often have extra space to shoot because multiple defenders are trying to make it hard for him to get a good look, or Rudy Gobert being on the court changes an opposing team's play selection because he's such a good shot blocker that plays that would lead to a bucket against a different player are leading to him getting a block so teams avoid those plays. Someone insistent on "objective measures" won't even consider these as things to potentially care about unless they have a way to measure them (which, now that we have sophisticated player-movement-tracking, we can actually measure things like how close the nearest defender is on shots by a Curry teammate when he's on court vs off court, or what percentage of opponents' shots are taken in a specific part of the court when Gobert is defending them vs when he isn't. So those measurements are coming online over time.) And, of course, understanding that it means something that Curry's teammates have extra space to shoot, or that Gobert's opponents might not be running their strongest offensive plays because of his shot blocking, puts us in the realm of meaning rather than mere measurement. Knowing to make the value judgment of "it matters how this player is impacting the game in ways we don't have a good numerical measurement of, but that a sophisticated observer who values those things can watch for and give subjective consideration of" puts us in the world of meaning rather than mere measurement.

That seems to be the same issue underlying this discussion. Knowing that conceptualizing a problem well matters -- and that it will profoundly impact the end result in objective areas even though it's not directly measured -- is wisdom.

[austin-cheney]

Ohhhhhh shit, that has to be the longest paragraph in all of HN. Have you seen the movie Moneyball?

Just start with the premise that bias isn’t helpful and even less helpful when implicit. Let that determine what to measure and will not be harmed. If such decisions twist you into knots then you are the person qualified to make such decisions.

[lotharbot]

> "bias isn’t helpful ... Let that determine what to measure"

Determining what counts as "bias" is itself a subjective activity. People who grow up in different cultures have different baselines for what factors matter the most and what factors they consider to be overvalued, undervalued, inappropriately accounted for, and so on. Not just different countries, but different subcultures within the same country (like, my cousins from the farm see a lot of things in society as biased toward big cities, which I never considered because I've lived in big cities for essentially my entire life.)

"start with the premise that bias isn't helpful" is, by the way, also not a measurable goal, which IMO supports what I'm saying. Knowing how to conceptualize a problem well (of which "eliminate bias" is a small subset) isn't something you can objectively measure, but it's something that will impact your objective measures down the line.