[sarosh]

The paper itself is here: https://arxiv.org/abs/2111.09259 with the key conclusion that "Examining the evolution of human concepts using probing showed that many human concepts can be accurately regressed from the AZ network after training, even though AlphaZero has never seen a human game of chess, and there is no objective function promoting human-like play or activations" and "[t]he fact that human concepts can be located even in a superhuman system trained by self-play broadens the range of systems in which we should expect to find human-understandable concepts"

[iechoz6H]

Aren't the shared concepts simply predicated on 'the rules of chess'? No surprise that an algorithm and a 'rational actor' share similar themes when both are constrained by the same rules.

[baq]

the concept of material isn't found in the rules and yet was discovered independently during self-play.

[Santosh83]

We are quite literally alpha zero ourselves. We developed chess by playing ourselves. I don't get what is particularly surprising that there are not infinite ways to play good chess?

[ogogmad]

It's not a priori obvious that a machine can learn to play chess in a similar way to the way a human can.

I suppose if AGI gets achieved then some of this will seem predictable in hindsight. But we don't yet know if it's achievable.

[V-2]

Human minds didn't evolve for playing chess. So there was a possibility that there are "hardware" limitations that prevent us from playng it the way it should be (give or take the inevitable error margin).

[kevinventullo]

The human-understandable concepts are there, but I think finding them is still very hard, right? I.e. it doesn’t sound like the system including AZ was able to identify and articulate the human concepts such as king safety a priori.

[iratewizard]

A game like chess is like two competing mathematical models. The human understanding of it is really just noticing patterns that exist in the model. I would hope an ML algorithm finds similar patterns.

[posterboy]

Can this be explained by the fact that the rules of chess evolved accomodating certain preferences that human players developed over the course of time?

Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?