> In spite of the diversity of strategies one can design, it is important to remark that the heads-up limit Texas hold'em variation has been claimed to be "essentially weakly solved" in January 2015 by the Cepheus poker-playing bot.[1] This means that on average the program is so good that a human would have no chance of ever edging ahead of it, even if the two played 60 million hands.[2] The bot can be played online at poker.srv.ualberta.ca, and users can even query strategies from the software
I watched that, and I think his understanding of what a "statistical tie" is is extremely dodgy (https://youtu.be/gz9FJfe2YGE?t=10m43s). What's up with that?
Edit: he does make a good point later on that playing thousands of games is exhausting, which would affect the reliability of the comparison between the AI and the pros. I don't know anything about poker, but it sounds plausible. Presumably this means they should make the AI so good that it can demonstrate its greatness quicker than a human gets tired.
Personally, I think it would be much more interesting to see an AI compete in a multi-player game with, say, ten human pros at the table. Not just heads-up.
There's so much variance in the game that it would be hard to know how good the AI really is. If they're doing 120K hands heads-up, they'd need to play far more in a full table. Getting that many pros to play that many hands would be difficult.
And what's wrong with heads-up, anyway? It's not as popular as multiway, but it's a fascinating game. In a full table game, you usually only end up playing 20-25% of your hands. Heads up you play 90%+. You end up in leveling wars with your opponent more often, too ("he knows that I know that he knows I usually wouldn't play a premium pair this way").
All you'd discover is that poker is a very very volatile game and the number of tournaments it would have to play in to prove anything would be beyond time available.
It's worth noting that most poker pros will constantly re-buy in early rounds in big tournaments. It's a general recognition that it takes money, not skill to survive. Being at the later tables helps them maintain their fame, which is really their business.
Of course, if you're playing against amateurs or drunks, you can always win. But usually at the real tournaments they are few and far between.
Oh, so playing online with access to statistical tools and analysis, so really the computer advantage for knowing the odds is completely nil. It's really only then competing in the strategy element... very interesting indeed.
I'd consider it unlikely (especially with two-tabling, and that poker players tend to study game theory and understanding of odds offline, not "live" or during gameplay itself) that the players will be referencing or accessing any tools during play.
It's just that it's becoming more and more in the toolbox of poker players to gain understanding of Nash equilibrium (conceptually and practically) and other important concepts that they fold into their strategy (based on meta-game, familiarity with the style and strategies of opponents, etc.)
More sophisticated players are constantly calculating real and implied odds at all times, without relying on a tool to do so (often there is fuzzing or approximation involved, but it's typically "close enough"), but especially in heads up play it's the more esoteric parts of strategy that are more important. Bluffing, slowplays/trapping, reads and reaction (not always physical as in tells but often habits).
Calculating odds against hand ranges is too complex to be done by a human in real time while playing, even using calculators made for the purpose. Computers definitely have an advantage there. Assigning hand ranges in the first place is the trickier part for an AI, as it is more rooted in psychology.
Sibs have alluded to this but it bears repeating: it's not a matter of simply calculating your "odds". One of the first things a player learns when getting serious about poker is how to calculate their odds of making a given hand, it becomes quite second nature, and even being able to work with floating points rather than rounded approximations would not give the AI a significant advantage.
As one sib points out, calculating your odds of winning against a range of hands is significantly more difficult, but again raw computation would not give a significant advantage here as assigning an opponent an accurate range of hands without significant historical data on their play is more or less impossible.
Any online poker pro will tell you that 120,000 hands is nothing. You can win over the first 120,000 hands and lose over the next 120,000 hands or break even. This is because the margins are low and the variance is very high. Someone winning 2 big blinds per 100 hands (with 80 bb variance) will win between 7901bb and -3141bb 95% of the time.
Also, it appears they are playing a tournament, which increases the variance significantly. I actually think tournament play is less suited to measuring AI performance than a cash game, because you're basically playing a series of different games as the tournament progresses that may not be generalizable because the decision math changes based on blinds and stack size unlike a cash game.
> To ensure that the outcome of the competition is not due to luck, the four pros will be paired to play duplicate matches — Player A in each pair will receive the same cards as the computer receives against Player B, and vice versa. One of the players in each of these pairs will play on the floor of the casino, while his counterpart will be isolated in a separate room.
This technique won't work for the same reason mentioned above. Hands in tournament poker are necessarily not independent events.
You can compute perfect strategies for much simpler variants of Hold’em, e.g. Leduc Hold’em. Then you see that bluffing is absolutely required for optimal play.
Bluffing seems like a human tactic but in fact is completely mathematically justified and is absolutely required for optimal play. A bot that bluffs will destroy a bot that doesn't.
I'm guessing they're training the AI independently for each player, with that in mind I wonder how good it would perform when matched against a different player. Knowing your opponent's playing style is one of the main things you have to master in poker. This is specially difficult as playing style will vary as the game goes, and sometimes solely by the player's will (to trick you) and not by some information derived from the game. I can see a bot performing reasonably well on this task for a specific player despite all the difficulties, but for the general case it seems like a huge challenge (as it is for humans).
Nobody who's been paying attention in the poker community would be shocked by this. HULHE has already been a AI win for a few years. HUNLHE is basically just a matter of time.
Nash equilibrium strategies specifically do work better against thinking opponents - the usual complaint about them is that they don't maximally exploit "bad" opponents.
I feel that Poker is one of those games that an AI could develop a winning strategy for, but not the most winningest. If you deployed this AI on pokerstars, it would win you money. If you deployed it at the WSOP, it would not win a bracelet.
nit: as a former professional poker player, the skill level required to be a long-term winning player at a high-stakes cash game is considerably higher than the skill requirement to win a single high-stakes tournament. (I would not voluntarily sit at a high-stakes cash table without knowing there were a mark present, I would voluntarily play most WSOP tournaments)
One thing to support this is the idea that there doesn't seem to be much more to differentiate a winner and a loser at the WSOP than randomness. Among the best players, they are likely so close in skill that winning and event is determined by a slightly weighted coin toss.
The other side is that if AI could discover some elements of lie detection that even humans cannot observe. It may not have anything to do with the power of the AI's computing, but more that its hardware can observe outside stimuli that humans cannot, and the AI may be able to integrate it into its strategy (e.g. a facial twitch that cannot even be seen by the naked eye).
Dwan is something of a reclusive/shady figure these days. Also it's unlikely he's currently playing anywhere close to his skill peak/ the skill level of the four selected - once a poker pro has achieved a level of fame commensurate to his, they can live comfortably for the rest of their days bleeding wealthy amateurs who get their kicks from playing (and losing) high stakes cash games against pros.
Don't know about Blom - probably just insufficient action for him (couldn't tell from the article, but it seems likely they're playing play money cash games, with fixed prizes for players who end ahead).
They're heads-up pros. Ivey and many other well-known names are not heads up specialists like these competitors. They'd likely have an edge over almost any "recognizable" name player.
The 90's called. They want their poker strategy back.
Poker is not about reading your opponents "tells". That is a myth perpetuated by Hollywood and ... professional poker players.
While there definitely could be some value in using psychological manipulation to best your "live" opponent, gaining some advantage via a physical tell is a crap shoot at best except against the absolute worst players or the biggest mark.
[+] [-] arikrak|9 years ago|reply
https://en.wikipedia.org/wiki/Heads_up_poker
[+] [-] Tenoke|9 years ago|reply
[+] [-] Iuz|9 years ago|reply
[+] [-] conistonwater|9 years ago|reply
Edit: he does make a good point later on that playing thousands of games is exhausting, which would affect the reliability of the comparison between the AI and the pros. I don't know anything about poker, but it sounds plausible. Presumably this means they should make the AI so good that it can demonstrate its greatness quicker than a human gets tired.
[+] [-] petters|9 years ago|reply
[+] [-] jim-greer|9 years ago|reply
And what's wrong with heads-up, anyway? It's not as popular as multiway, but it's a fascinating game. In a full table game, you usually only end up playing 20-25% of your hands. Heads up you play 90%+. You end up in leveling wars with your opponent more often, too ("he knows that I know that he knows I usually wouldn't play a premium pair this way").
[+] [-] bluetwo|9 years ago|reply
[+] [-] RockyMcNuts|9 years ago|reply
[+] [-] blazespin|9 years ago|reply
It's worth noting that most poker pros will constantly re-buy in early rounds in big tournaments. It's a general recognition that it takes money, not skill to survive. Being at the later tables helps them maintain their fame, which is really their business.
Of course, if you're playing against amateurs or drunks, you can always win. But usually at the real tournaments they are few and far between.
[+] [-] andrewclunn|9 years ago|reply
[+] [-] jat850|9 years ago|reply
It's just that it's becoming more and more in the toolbox of poker players to gain understanding of Nash equilibrium (conceptually and practically) and other important concepts that they fold into their strategy (based on meta-game, familiarity with the style and strategies of opponents, etc.)
More sophisticated players are constantly calculating real and implied odds at all times, without relying on a tool to do so (often there is fuzzing or approximation involved, but it's typically "close enough"), but especially in heads up play it's the more esoteric parts of strategy that are more important. Bluffing, slowplays/trapping, reads and reaction (not always physical as in tells but often habits).
[+] [-] danenania|9 years ago|reply
[+] [-] MaxfordAndSons|9 years ago|reply
As one sib points out, calculating your odds of winning against a range of hands is significantly more difficult, but again raw computation would not give a significant advantage here as assigning an opponent an accurate range of hands without significant historical data on their play is more or less impossible.
[+] [-] morkalt|9 years ago|reply
[deleted]
[+] [-] iopq|9 years ago|reply
[+] [-] asher_|9 years ago|reply
Also, it appears they are playing a tournament, which increases the variance significantly. I actually think tournament play is less suited to measuring AI performance than a cash game, because you're basically playing a series of different games as the tournament progresses that may not be generalizable because the decision math changes based on blinds and stack size unlike a cash game.
> To ensure that the outcome of the competition is not due to luck, the four pros will be paired to play duplicate matches — Player A in each pair will receive the same cards as the computer receives against Player B, and vice versa. One of the players in each of these pairs will play on the floor of the casino, while his counterpart will be isolated in a separate room.
This technique won't work for the same reason mentioned above. Hands in tournament poker are necessarily not independent events.
[+] [-] scotchio|9 years ago|reply
Edit: Yes.
> During last year’s contest, the pros noticed Claudico was making some all-too-obvious bluffs that they were able to exploit.
[+] [-] danenania|9 years ago|reply
Some pros use systems where they look at the second hand of a clock to decide whether to bluff or not, as a proxy for randomness.
[+] [-] petters|9 years ago|reply
[+] [-] dfan|9 years ago|reply
[+] [-] user5994461|9 years ago|reply
[+] [-] necessity|9 years ago|reply
[+] [-] conistonwater|9 years ago|reply
[+] [-] rampage101|9 years ago|reply
The pure Nash Equilibrum strategies do not work that well against thinking opponents.
[+] [-] splonk|9 years ago|reply
Nash equilibrium strategies specifically do work better against thinking opponents - the usual complaint about them is that they don't maximally exploit "bad" opponents.
[+] [-] petters|9 years ago|reply
But yes, that would not be a good idea since optimal play is a complicated mixed strategy.
[+] [-] spectrum1234|9 years ago|reply
I played poker professionally for 7 years (until banned in US) and am still amazed by this.
[+] [-] wapz|9 years ago|reply
[+] [-] mmanfrin|9 years ago|reply
[+] [-] jknoepfler|9 years ago|reply
[+] [-] kolbe|9 years ago|reply
The other side is that if AI could discover some elements of lie detection that even humans cannot observe. It may not have anything to do with the power of the AI's computing, but more that its hardware can observe outside stimuli that humans cannot, and the AI may be able to integrate it into its strategy (e.g. a facial twitch that cannot even be seen by the naked eye).
[+] [-] return0|9 years ago|reply
[+] [-] jat850|9 years ago|reply
[+] [-] IamHWengineer|9 years ago|reply
[+] [-] curun1r|9 years ago|reply
[+] [-] Mikeb85|9 years ago|reply
[+] [-] MaxfordAndSons|9 years ago|reply
Don't know about Blom - probably just insufficient action for him (couldn't tell from the article, but it seems likely they're playing play money cash games, with fixed prizes for players who end ahead).
[+] [-] prawn|9 years ago|reply
[+] [-] lintiness|9 years ago|reply
[+] [-] jat850|9 years ago|reply
[+] [-] TylerE|9 years ago|reply
[+] [-] runeks|9 years ago|reply
[+] [-] freebeeme|9 years ago|reply
[deleted]
[+] [-] CoryG89|9 years ago|reply
[+] [-] KODeKarnage|9 years ago|reply
Poker is not about reading your opponents "tells". That is a myth perpetuated by Hollywood and ... professional poker players.
While there definitely could be some value in using psychological manipulation to best your "live" opponent, gaining some advantage via a physical tell is a crap shoot at best except against the absolute worst players or the biggest mark.