Sounds like when you teach a neural network chess, it ends up learning many of the same concepts that we do.
To me, this is a great illustration that superhuman intelligence is not magic. When a superhuman AI plays chess, the moves usually make sense to an expert. And even if they don't immediately make sense, they usually make sense once the expert plays out some lines to see what happens. Superhuman AIs have crushed humans at chess not by discovering new, wildly counterintuitive openings that dumb humans missed (it actually plays many of the same openings). Rather, it just does what humans do - win pieces, keep your king safe, attack, gain space, restrict your opponent's moves - but better and more consistently.
This paper builds on that concept and finds that not only are a superhuman AI's moves usually understandable by experts, but even some of the superhuman AI's internal representations are understandable by experts (!).
In the 5th century BCE, Greek philosophers thought the Earth was a sphere. Although they were eventually improved upon by Newton's ellipsoid, they still had arrived at the right concept. And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding still builds upon the same concepts.
Thinking broadly, like, suppose we invent some superhuman AGI for $100B in 2100. And then we ask it how to optimize our paperclip production. I don't think its recommendations will be magic. Paperclip production is a fairly well understood problem: you bring in some metal, you reshape it, you send it out. It's probably gonna have much of the same advice we've already figured out: invest in good equipment, take advantage of economies of scale, choose cheap but effective metals, put factories near sources of energy and labor and raw materials, ship out paperclips in a hierarchical network with caches along the way to supply unanticipated demand, etc. Like, an AGI's suggestions may well be better than a management consultant's, but it's not like it's going to wave a magic wand and flip the laws of physics. Rather, I expect it to win within the same frameworks we've developed, but by doing things better and more consistently than humans.
> When a superhuman AI plays chess, the moves usually make sense to an expert. And even if they don't immediately make sense, they usually make sense once the expert plays out some lines to see what happens.
Do you actually play chess? Half the moves made by an AI at 3500+ ELO (e.g. Stockfish 13+) is completely alien to even a human super GM like Nakamura or Carlsen. It's actually the most reliable and used way to know whether someone is cheating or not, if the moves don't make sense to high level players, then it's very probably someone using an engine.
> And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding is still leverages the same concepts.
That is very far from being a truth that lots of chess experts would agree on. Most chess players at high level reason with similar, well established concepts, such as tempo, positions, baits, color dominance, etc.
Super AIs don't use these concepts, and it's obvious when you see their moves. They often break tempo abruptly, disregard color dominance, enter what looks like dangerous positions,etc.
> Rather, it just does what [less successful people] do - ... - but better and more consistently.
Reminds me of comparisons between Renaissance and other quant funds. If you get enough better it looks like what you're doing is magic until you analyse what was done and see they have better systems and people.
Sometimes, magic is just someone spending more time on something than anyone else might reasonably expect - Teller
After watching some of the games AlphaZero played against Stockfish, I have to say I don't really agree that the moves make sense to a grandmaster... AlphaZero has this uncanny ability to sacrifice many pawns to restrict the movement of the opponent's pieces, something that you might see in advanced play but nowhere near the extent you see it with AlphaZero. Many more of the moves it plays are hard to make sense of compared to a classic engine such as Stockfish.
> When a superhuman AI plays chess, the moves usually make sense to an expert.
It's really interesting to look at very high level players analyzing specific AI moves. Many they understand after a short while, others take longer - and some they don't understand directly, but might have an inkling as to why it's good.
My feeling is that many high level analysts would rather just automatically defer to the AI's analysis rather than challenging it - so there is in some sense a lack of feedback to correct such situations. Even in the 1997 Deep Blue vs Kasparov match, game #1 Deep Blue apparently was a bugged position so it moved randomly (a legal random move) - but because Kasparov believed it was correct he felt the analysis was way beyond him so he conceded.
But could there be a class of games whose solutions are so complex that we can't understand them, yet an AGI would?
We designed chess to be easily understood by humans. There's a cap on the conceptual complexity that fits within our capabilities. So it's not surprising that we can understand AI solutions that merely asymptote to that fixed cap.
Chess are very limited compared to the whole universe, AGI could for example invent self-replicating fusion-driven paperclip factories, there's nothing in the universe rules that makes this impossible we simply don't know how to do it (yet).
Serious question: I haven't done the research but I've always felt that in abstracts like Chess and Go that a larger part of the skill is in "reading the board". Humans are leveraging our pattern matching but we can get tired or miss even one little thing. The AI never misses, ever. I've always wondered how a human would do against an AI if the human was given a list of the top 50 potential moves in a randomized order to consider before making their move. Would the human be on more level playing field in terms of reading?
The fallacy here is that we can only measure whether AZ is playing human strategies, by comparing against our own known space of strategies. Sure, we human have developed good heuristics the AI has also found, but we can’t know if it has better/more subtle concepts that we haven’t discovered, exactly because we haven’t discovered them yet. Who is to say “mobility” is even a coherent top level concept compared to some unpronounceable abstraction the AI has figured out?
In the first match of the Lee Sedol / Alpha Go jubango, many moves were initially interpreted as slow, or simply mistakes.
Human players try to maximize score difference, Alpha Go tries to maximize the probability of winning. This means, between playing a risky move that makes more territory, and a safe move that makes less territory, Alpha Go prefers the safer move.
To experts, those move came across as "slow", inefficient or suboptimal. But they turn out to be the superior moves.
The paper itself is here: https://arxiv.org/abs/2111.09259 with the key conclusion that "Examining the evolution of human concepts using probing showed that many human concepts can be accurately regressed from the AZ network after training, even though AlphaZero has never seen a human game of chess, and there is no objective function promoting human-like play or activations" and "[t]he fact that human concepts can be located even in a superhuman system trained by self-play
broadens the range of systems in which we should expect to find human-understandable concepts"
Aren't the shared concepts simply predicated on 'the rules of chess'? No surprise that an algorithm and a 'rational actor' share similar themes when both are constrained by the same rules.
We are quite literally alpha zero ourselves. We developed chess by playing ourselves. I don't get what is particularly surprising that there are not infinite ways to play good chess?
The human-understandable concepts are there, but I think finding them is still very hard, right? I.e. it doesn’t sound like the system including AZ was able to identify and articulate the human concepts such as king safety a priori.
Can this be explained by the fact that the rules of chess evolved accomodating certain preferences that human players developed over the course of time?
Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?
The sci-fi trope that aliens or superintelligent AIs will have thought processes completely different and incomprehensible to humans is probably not true.
If the aliens live in our universe, evolved on a planet with limited resources, need food, breathe, etc., then they'll likely have a lot in common with Earth animals psychologically. If the AI is tasked with doing something that humans also do, then its tactics will be at least recognizable to an expert in that field. Moreso in well-understood systems such as chess.
Is it really plausible that an advanced chess AI's tactics would have no concept of threats or king safety? The same tactics apply to it as do to us.
I don't want to be too dismissive of the work going into this, but Chess isn't that huge of a search space. Games can quickly end in many scenarios from certain points.
Does having an evaluation system that takes into account king safety only as a consequence of having taken the entire game state into account, mean that it understands king safety? Can you extract these tactics from the AI? Can it explain the moves?
I would be interested in seeing work here that can provide some comprehensibility to the correlations, as that's how we can get to actual novel intuition. "The computer is good with this neural network due to pattern matching" is less actionable than "the computer is good and found this kind of pattern, and we think it relates to this".
EDIT: I think that if anything Chess is an interesting example of humans emulating computers. People sitting around thinking about variations, trying to snuff out what is good/bad. Granted, the fact that you're playing against other humans is pretty important at a higher level, but it's centuries of people sitting down and running through the search space, using our approximations for what are good and bad.
I don't share this view. They would have had a lot in common at a point in time in their history, but who's to say the consequences of millions of years of additional evolution post-energy abundance, let alone the consequences of embryo selection and genetic engineering? We've had a few tens of thousand years of evolution with relative (but still not really) caloric abundance and look how much bigger our brains got. Add nuclear fusion and an extra 50 million years to that nascent process.
I think chess strategy is a flawed way to be thinking about this because we designed the rules and the solution space to be comprehensible to us. It's not surprising that we can understand AI strategy. That simply follows from the solution space itself being small and comprehensible.
Chess is a closed system. Alien beings could be "playing checkers", so to speak. A biology completely different to anything existing on Earth (say, inteligent life living in the astmophere of a hot gas giant planet) would likely lead an utterly incompatible psychology. It's kind of like in the famous Wittgenstein quote: "if a lion could speak, we could not understand him". Well, lion is an Earth animal, but you see the point.
> then they'll likely have a lot in common with Earth animals psychologically
The term "psychologically" here is interesting because it skips the "biologically" part. What if they don't have neurons like us or AZ? Could one even talk about psychology?
I think it pretty cool how AlphaZero does not converge to a single best opening move and its variation of opening moves is higher than humans.
Seeing people analyze AlphaZero games on YouTube, it plays a more open and aggressive game of chess and goes for the win instead of the very defensive play of most of the top players today. Some people hope that if people study how AlphaZero plays chess (the best chess player in the world) to improve their game it might make chess more interesting as most games at the high level now end in draws.
It's a somewhat counterintuitive result from game theory that many of the best strategies we know to various non-trivial games are "mixed strategies". This refers to a strategy (presumably) like the one by AlphaZero here, where moves are chosen from a probabilistic distribution, rather than just picking "the best one". The sub-optimal move choice is counterbalanced by the unpredictability.
The Chessbase article is a lot more readable than the arxiv paper. Yeah, they did some bad GPL stuff, but that just isn't really relevant to this particular discussion, and the chessbase link is better for the general reader.
> Why link to Chessbase instead of to the arxiv paper directly?
Because their article is easier to read, your link isn't even directly to the paper.
And unlike the fake news media who gate-keeps information to ensure their monopoly and to stop you Gell-Mann Amnesia them, Chessbase has included a direct link to the PDF and the abstract at the bottom of their article. That's awesome.
Bike-shedding GPL violations over quality and honest articles doesn't interest lots of people. They can read about the OT stuff in proper HN discussions on it.
>AlphaZero’s neural network evaluation function doesn’t have the same level of structure as Stockfish’s evaluation function: the Stockfish function breaks down a position into a range of concepts (for example king safety, mobility, and material) and combines these concepts to reach an overall evaluation of the position. AlphaZero, on the other hand, outputs a value function ranging from -1 (defeat is certain) to +1 (victory is guaranteed) with no explicitly-stated intermediate steps. Although the neural network evaluation function is computing something, it’s not clear what.
I find it fascinating that we can create something that can "learn" in some sense in a way we don't fully understand. Or am I misunderstanding the meaning of this passage?
The way in which I think about it is that "learning" (in humans) often occurs is in the form of heuristics - certain rules or shortcuts which are not guaranteed to be optimal, but do a good job at simplifying (the search-state of) the problem. Instead of trying many simple comparisons between legal moves, we instead use a heuristic to score them.
Non-NN chess engines are really all about doing a good job at balancing these heuristics (like it says above). There's fundamentally an upper bound in how good the overall approach can be, simply because the underlying heuristics are defined and selected by humans: there could be many more heuristics that haven't been considered. This version of "learning" is really just removing human forced constraints which were introduced to help learning, by starting from a position without bias.
We do understand how they learn, but the structure of their knowledge is currently hard to discern in the same way that you can't get thoughts or memories out of somebody's head with a MRI scan.
Yes, it has been done with go and Deep Mind. The program found novel patterns of playing that were not known. I don't see why this wouldn't happen with chess.
>From recorded data, we can see that everyone seemed to play e4 in the 1500s. Over centuries, moves like d4, Nf3 or c4 emerged as credible and fashionable alternatives.
The graph they have seems to suggest the opposite - nobody playing e4, just d4 early on (and AZ also gravitating most towards d4).
[+] [-] tedsanders|4 years ago|reply
To me, this is a great illustration that superhuman intelligence is not magic. When a superhuman AI plays chess, the moves usually make sense to an expert. And even if they don't immediately make sense, they usually make sense once the expert plays out some lines to see what happens. Superhuman AIs have crushed humans at chess not by discovering new, wildly counterintuitive openings that dumb humans missed (it actually plays many of the same openings). Rather, it just does what humans do - win pieces, keep your king safe, attack, gain space, restrict your opponent's moves - but better and more consistently.
This paper builds on that concept and finds that not only are a superhuman AI's moves usually understandable by experts, but even some of the superhuman AI's internal representations are understandable by experts (!).
In the 5th century BCE, Greek philosophers thought the Earth was a sphere. Although they were eventually improved upon by Newton's ellipsoid, they still had arrived at the right concept. And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding still builds upon the same concepts.
Thinking broadly, like, suppose we invent some superhuman AGI for $100B in 2100. And then we ask it how to optimize our paperclip production. I don't think its recommendations will be magic. Paperclip production is a fairly well understood problem: you bring in some metal, you reshape it, you send it out. It's probably gonna have much of the same advice we've already figured out: invest in good equipment, take advantage of economies of scale, choose cheap but effective metals, put factories near sources of energy and labor and raw materials, ship out paperclips in a hierarchical network with caches along the way to supply unanticipated demand, etc. Like, an AGI's suggestions may well be better than a management consultant's, but it's not like it's going to wave a magic wand and flip the laws of physics. Rather, I expect it to win within the same frameworks we've developed, but by doing things better and more consistently than humans.
[+] [-] Galanwe|4 years ago|reply
Do you actually play chess? Half the moves made by an AI at 3500+ ELO (e.g. Stockfish 13+) is completely alien to even a human super GM like Nakamura or Carlsen. It's actually the most reliable and used way to know whether someone is cheating or not, if the moves don't make sense to high level players, then it's very probably someone using an engine.
> And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding is still leverages the same concepts.
That is very far from being a truth that lots of chess experts would agree on. Most chess players at high level reason with similar, well established concepts, such as tempo, positions, baits, color dominance, etc.
Super AIs don't use these concepts, and it's obvious when you see their moves. They often break tempo abruptly, disregard color dominance, enter what looks like dangerous positions,etc.
[+] [-] barry-cotter|4 years ago|reply
Reminds me of comparisons between Renaissance and other quant funds. If you get enough better it looks like what you're doing is magic until you analyse what was done and see they have better systems and people.
Sometimes, magic is just someone spending more time on something than anyone else might reasonably expect - Teller
[+] [-] ucha|4 years ago|reply
See GothamChess' very entertaining analysis: https://www.youtube.com/watch?v=8dT6CR9_6l4
[+] [-] noxvilleza|4 years ago|reply
It's really interesting to look at very high level players analyzing specific AI moves. Many they understand after a short while, others take longer - and some they don't understand directly, but might have an inkling as to why it's good.
My feeling is that many high level analysts would rather just automatically defer to the AI's analysis rather than challenging it - so there is in some sense a lack of feedback to correct such situations. Even in the 1997 Deep Blue vs Kasparov match, game #1 Deep Blue apparently was a bugged position so it moved randomly (a legal random move) - but because Kasparov believed it was correct he felt the analysis was way beyond him so he conceded.
[+] [-] csee|4 years ago|reply
We designed chess to be easily understood by humans. There's a cap on the conceptual complexity that fits within our capabilities. So it's not surprising that we can understand AI solutions that merely asymptote to that fixed cap.
[+] [-] ajuc|4 years ago|reply
[+] [-] snarf21|4 years ago|reply
[+] [-] themeiguoren|4 years ago|reply
[+] [-] 29athrowaway|4 years ago|reply
Human players try to maximize score difference, Alpha Go tries to maximize the probability of winning. This means, between playing a risky move that makes more territory, and a safe move that makes less territory, Alpha Go prefers the safer move.
To experts, those move came across as "slow", inefficient or suboptimal. But they turn out to be the superior moves.
[+] [-] sarosh|4 years ago|reply
[+] [-] iechoz6H|4 years ago|reply
[+] [-] Santosh83|4 years ago|reply
[+] [-] kevinventullo|4 years ago|reply
[+] [-] posterboy|4 years ago|reply
Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?
[+] [-] a_shovel|4 years ago|reply
If the aliens live in our universe, evolved on a planet with limited resources, need food, breathe, etc., then they'll likely have a lot in common with Earth animals psychologically. If the AI is tasked with doing something that humans also do, then its tactics will be at least recognizable to an expert in that field. Moreso in well-understood systems such as chess.
Is it really plausible that an advanced chess AI's tactics would have no concept of threats or king safety? The same tactics apply to it as do to us.
[+] [-] rtpg|4 years ago|reply
Does having an evaluation system that takes into account king safety only as a consequence of having taken the entire game state into account, mean that it understands king safety? Can you extract these tactics from the AI? Can it explain the moves?
I would be interested in seeing work here that can provide some comprehensibility to the correlations, as that's how we can get to actual novel intuition. "The computer is good with this neural network due to pattern matching" is less actionable than "the computer is good and found this kind of pattern, and we think it relates to this".
EDIT: I think that if anything Chess is an interesting example of humans emulating computers. People sitting around thinking about variations, trying to snuff out what is good/bad. Granted, the fact that you're playing against other humans is pretty important at a higher level, but it's centuries of people sitting down and running through the search space, using our approximations for what are good and bad.
[+] [-] TheRealNGenius|4 years ago|reply
[+] [-] csee|4 years ago|reply
I think chess strategy is a flawed way to be thinking about this because we designed the rules and the solution space to be comprehensible to us. It's not surprising that we can understand AI strategy. That simply follows from the solution space itself being small and comprehensible.
[+] [-] V-2|4 years ago|reply
[+] [-] astrobe_|4 years ago|reply
The term "psychologically" here is interesting because it skips the "biologically" part. What if they don't have neurons like us or AZ? Could one even talk about psychology?
[+] [-] njarboe|4 years ago|reply
Seeing people analyze AlphaZero games on YouTube, it plays a more open and aggressive game of chess and goes for the win instead of the very defensive play of most of the top players today. Some people hope that if people study how AlphaZero plays chess (the best chess player in the world) to improve their game it might make chess more interesting as most games at the high level now end in draws.
[+] [-] Y_Y|4 years ago|reply
[+] [-] porphyra|4 years ago|reply
Chessbase is notoriously sketchy and there is a lawsuit against them for blatantly stealing GPL code.
https://stockfishchess.org/blog/2021/our-lawsuit-against-che...
[+] [-] lacker|4 years ago|reply
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] aaron695|4 years ago|reply
Because their article is easier to read, your link isn't even directly to the paper.
And unlike the fake news media who gate-keeps information to ensure their monopoly and to stop you Gell-Mann Amnesia them, Chessbase has included a direct link to the PDF and the abstract at the bottom of their article. That's awesome.
Bike-shedding GPL violations over quality and honest articles doesn't interest lots of people. They can read about the OT stuff in proper HN discussions on it.
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] AutumnCurtain|4 years ago|reply
I find it fascinating that we can create something that can "learn" in some sense in a way we don't fully understand. Or am I misunderstanding the meaning of this passage?
[+] [-] noxvilleza|4 years ago|reply
Non-NN chess engines are really all about doing a good job at balancing these heuristics (like it says above). There's fundamentally an upper bound in how good the overall approach can be, simply because the underlying heuristics are defined and selected by humans: there could be many more heuristics that haven't been considered. This version of "learning" is really just removing human forced constraints which were introduced to help learning, by starting from a position without bias.
[+] [-] beaconstudios|4 years ago|reply
[+] [-] phreeza|4 years ago|reply
[+] [-] karol|4 years ago|reply
[+] [-] Tenoke|4 years ago|reply
The graph they have seems to suggest the opposite - nobody playing e4, just d4 early on (and AZ also gravitating most towards d4).
[+] [-] redsummer|4 years ago|reply
[deleted]