[eggsnbacon1]

I think some bias may be inherent.

To blue eyes again, you could make a facial recognition system that just looked for 2 blue iris shapes and probably score 90%+ accuracy on most datasets (of exclusively blue eyed people and non-human objects) because blue is so rare in animal eyes, and so rare in nature. With such a strong face signal it may be impossible to match accuracy for non-blue-eyes humans without purposely making match quality for blue eyes worse.

We may be seeing a similar but weaker bias for darker skin. Light skin is fairly rare among mammals. The only solution to eliminating recognition accuracy bias may be to nerf accuracy for people with light skin. There have been many reports of facial recognition erroneously classifying dark skinned humans as animals. If part of model accuracy classifies lighter skin animal = more likely human, its going to be very hard to remove that bias because its empirically true. If the classifier is unsure if a shape is a human or animal face, but its very lightly colored, it may make the model statistically more accurate to report higher confidence that a lighter colored face is more likely to be human.

this may be more of a "most animals are brown" than a "brown people look more like animals" problem with the models. A possible path to fixing the bias is to crop all faces detected, disregarding model confidence of whether the face is human