[XuMiao]

Supervised learning algorithms assume that the input data are iid of the future. This is not valid in most of the real applications. The observation that we see men more than women in programming does not necessarily generalize to the future. That's why online learning provides an exploitation vs exploration mechanism to minimize the bias in the hindsight. In many applications, people just forgot about this simple strategy and blame the bias caused by supervised learning to the black box model.

Of course, black box AI itself is not the right solution. As more and more cross domain multitask settings emerge, open box AI will gradually take off. It is about compositional capability like functor and monad in functional language. Explanable or not is just a communication problem which is parallel to the ultimate intelligence problem. It is very possible that human intelligence is bounded.