[torinmr]

A particularly hairy case is where you use variables as features which are themselves generated through other potentially-discriminatory processes. For example, including a "has committed a felony" feature seems like a no-brainer for a hiring or lending application, but now any racial or other discrimination present in the criminal justice system has now "infected" your hiring process, so that the outcomes of your hiring process are now racially biased even though your process itself was not.

[nickpeterson]

But at some level, doesn't that just mean reality is biased, and since we don't live in utopia, trying to model one in the small is a sandcastle?

[inimino]

What do you mean by "reality"? Society is pervasively biased, yes, that's the point. That makes it difficult to fight against that bias, but it is necessary if you want to live in a more equal world.