[ASpring]

I wrote about this exact topic a few years back: "Algorithmic Bias is Not Just Data Bias" (https://aaronlspringer.com/not-just-data-bias/).

I think the author is generally correct but there is a lot of focus on algorithmic design and not on how we collectively decide what is fair and ethical for these algorithms to do. Right now it is totally up to the algorithm developer to articulate their version of "fair" and implement it however they see fit. I'm not convinced that is a responsibility that belongs to private corporations.

[fennecfoxen]

> I'm not convinced that is a responsibility that belongs to private corporations.

Private corporations are, by and large, the entities which execute their business using these algorithms, which their employees write.

They are already responsible for business decisions whether made using computers or otherwise. Indeed, who else would possibly manage such a thing? This is tantamount to saying that private corporations should have no business deciding how to execute their business — definitely an opinion you can have, it's just that it's an incredibly statist-central-planning opinion the end.

[naasking]

> Indeed, who else would possibly manage such a thing? This is tantamount to saying that private corporations should have no business deciding how to execute their business

No business is allowed to discriminate against protected groups. That's arguably a third-party standard for fairness, but I don't think this qualifies as central planning.

I see no reason why other types of third-party standards would be impossible or infeasible for machine learning applications.

[bluesummers5651]

One of the first papers I read in this area was very interesting in this regard (https://crim.sas.upenn.edu/sites/default/files/2017-1.0-Berk...). I think the challenge is that a business (e.g. COMPAS) can certainly take a position on what definition of algorithmic fairness they want to enforce, but the paper mentions six different definitions of fairness, which are impossible to satisfy simultaneously unless base rates are the same across all groups (the "data problem"). Even the measurement of these base rates itself can be biased, such as over- or under-reporting of certain crimes. And even if you implement one definition, there's no guarantee that that is the kind of algorithmic fairness that the government/society/case law ends up interpreting as the formal mathematical instantiation of the written law. Moreover, this interpretation can change over time since laws, and for that matter, moral thinking, also change over time.

I think the upshot to me is that businesses, whether it's one operating in criminal judicial risk assessment or advertising or whatever, don't really make obvious which definition (if any) of fairness that they are enforcing, and thus it becomes difficult to determine whether they are doing a good job at it.

[ASpring]

Maybe I wasn't very clear, I don't think every single machine learning model should be subject to regulation.

Rather I view it more along the lines of how the US currently regulates accessibility standards for the web or enforces mortgage non-discrimination in protected categories. The role of government here is identify a class of tangible harms that can result from unfair models deployed in various contexts and to legislate in a way to ensure those harms are avoided.

[ZeroGravitas]

I wonder what the countermeasures to this are.

If you trained a model to predict the outcome purely from the protected class and it was successful (in terms of predictive power), does that mean fairness is efectively impossible?

e.g. if you trained an educational performance predictor on wealth of parents, then I'd guess it would do reasonably well. And there is the argument that your parents are rich because they're smart and you are genetically connected to them.

But there's obvious counterexamples, like children adopted by rich families or children of refugees (who may have been professors or surgeons in their home country).

So if we can't avoid the bias in that extreme example, then adding extra data is only going to bury that truth under confusion.

I'm not sure we're ready to admit that we disadvantage the children of the poor to ourselves, which will make this whole AI bias thing a tricky conversation to have.