[valesco]

European law foresees those exact cases and IIRC prohibits automated treatment of personal information that leads to decisions that affects someone’s life, if no human takes the final decision and if the specific elements for the decision cannot be explained. In my opinion this forbids black boxes.

[dgellow]

What about credit score systems like the German SCHUFA? https://en.wikipedia.org/wiki/Schufa.

They have their proprietary algorithm to calculate a credit score, for example you almost cannot find an apartment without presenting your SCHUFA review and there is now way to know how their score is calculated (it has even been shown in the past that they use incorrect or outdated data for their calculation).

That’s the definition of a black box in my book.

[cauthon]

I think you’re referring to GDPR article 22, which prohibits decisions based solely on automated processing but imposes no constraints on the transparency of such processing. [1]

[1] https://gdpr-text.com/read/article-22/