By "unbeatable" they mean it lasted a few hours in the hands of the public before the hardcoded behaviors were exploited and it was defeated consistently. Turns out hardcoding the mechanism for players to manually block creeps (a more "creative" play style, but a fundamental element of lane control) allowed for the exploit. The bot did not teach itself to control the flow of creeps to gain a laning advantage against the opponent.
Not to diminish the teams' achievement. This was awesome. But again, it's just more over-hyping of AI that should be called out for what it really is.
They neither call it unbeatable nor mention hardcoding anything. The "unbeatable" is a quote from a professional tester, and they discuss the losses, acknowledging the need for further improvements for more general play.
Yeah, the more I learn about this the less I'm impressed. Seems like they wanted to steal DeepMind's/Blizzard's thunder and hacked together a demo that isn't quite as advanced as they made it out to be.
This is a highly editorialized title that clearly doesn't follow the posting guidelines:
> Otherwise please use the original title, unless it is misleading or linkbait.
> Please don't do things to make titles stand out ...
(the original title is the much more sedate "More on DOTA 2"; in case it is changed by moderators, at the time of this comment it says "How We Built the Unbeatable DOTA 2 bot", which is not even a quotation from the article)
The submitted title has certainly shaped the commentary (see people reacting to "unbeatable", and whether it's an impressive achievement, vs reacting to the technical content).
So, in a nutshell, I think we should avoid titles like this, and I believe the guidelines transparently say so as well.
This is an unquestionable technical achievement, but its important to keep in mind that the AI view of the world is through the bot API. This is compared to a human that decodes game state from observing the monitor. Not being an avid DOTA2 player I marvel at skilled players that can follow the game based on what looks like a very busy and colorful mess to me.
At the same time most of the deep learning field has been focusing on implementing super-human perceptual abilities (vision, hearing, translation) that come instinctually to humans. The higher level reasoning and memory/attention augmented machine learning is still cutting edge research. I think DeepMind and OpenAI are driving research towards that end.
As a developer I surely am amazed of the capabilities of the bot, but as a Dota player I am a bit more skeptic.
The bot only wins in a certain scenario, Shadow Fiend vs Shadow Fiend, no bottle, no runes, no jungle creeps and so on.
This cuts down the possibilities of playstyle a lot. This is stuff the pros are used to have available, and are training with. The bot was trained without it and has a fitting playstyle. For this certain type of 1v1, a pro would have to adapt. I still think that if you give them enough time, they will figure it out.
They didn't come up with the format by themselves. This is the "standard" 1v1 format. But yes as they acknowledged getting the bot to play 5v5 is a much bigger challenge.
The first step in the project was figuring out how to run Dota 2 in the cloud on a physical GPU. The game gave an obscure error message on GPU cloud instances. But when starting it on Greg’s personal GPU desktop (which is the desktop brought onstage during the show), we noticed that Dota booted when the monitor was plugged in, but gave the same error message when unplugged. So we configured our cloud GPU instances to pretend there was a physical monitor attached.
Dota didn’t support custom dedicated servers at the time, meaning that running scalably and without a GPU was possible only with very slow software rendering. We then created a shim to stub out most OpenGL calls, except the ones needed to boot.
If you watch the vids and read the blog posts and re watch it and reread it. You end up with just more and more questions. The bot learned to raze from the Fog Of War based on playing a pro once or twice? It changed its build based on losing once? (sure you say you white listed it but it still had choices no?). Wand not being a common opening is also not true. It's common AGAINST heavy casters. (I wont rush wand AS a bat rider but I would AGAINST him. I would AGAINST shadowfiend as well.) Was the animation canceling also taught similar to the creep block? independent RL?).
So many things don't fit in.. but that might be because I am neither good at DOTA 2 nor at AI stuff. Good luck with the 5v5's openai. And if the AI bots become public, maybe i'll get to play against them some day.
> It changed its build based on losing once? (sure you say you white listed it but it still had choices no?).
What I think happened here: the bot lost against someone using an item that hadn't been whitelisted before. The devs decided to include the item among the possible choices for self-training. Exploring the new choice, the bot adapted to using it to defeat its previous strategy. As a result, it now had to change its strategy to account for an opponent using the item.
Those who are thinking that AI is being overly hyped, and that the demos like these are not smart enough, think about it, today's AI is smarter enough IMO, as the potential of real-world applications of AI as of now itself could be vast. It just need some practical polished implementations. Did the Dota 2 bot learn the game from scratch by learning the rules all by itself? No, as evident from the post itself - "small amounts of “coaching” with self-play. But think about it. It need not be that smart to replace humans. IMO, many of our jobs do not need to very complicated self-learning. In real world day-day jobs like agriculture, factories, and to a great extend what we call as skilled labour like driving, management etc, the kind of work is often repetitive. AI is already better than humans at recognising speech, vision and to some extend languages. Combining these skills with mechanical robots in a meaningful way can easily replace what we, as humans can do, except in niche fields. Even there computer can offload our job in several ways. Robots/AI can do it much much faster, and much more efficient than us and it will only be getting better and better with overall advancement in research and the amount of data available..
So it's kind of interesting and equally scary that we are looking at a potential future, possibly near, where most of the work can be handled by a trained robot with humans only supervising them in exceptional cases. Robots can then could be only need to be smart enough to alert the human supervisors when they reach a state where they cannot perform the job normally, for e.g. power grid going off in a factory and they couldn't figure out what to do, it could only be a 0.001 % chance. The point is AI need not get as good as humans in learning complex scenarios to replace majority of us, it just need to do 99% of what we repeatedly do, much better than us.
Standard---maybe not objection, but complication: I'm waiting for the day AI can do this in 2100 KWh. (Human brain uses ~15W continuous. 15 W * 24h * 365 days * 16 years.)
[+] [-] crsv|8 years ago|reply
Not to diminish the teams' achievement. This was awesome. But again, it's just more over-hyping of AI that should be called out for what it really is.
[+] [-] sambe|8 years ago|reply
[+] [-] skgoa|8 years ago|reply
[+] [-] _dps|8 years ago|reply
> Otherwise please use the original title, unless it is misleading or linkbait.
> Please don't do things to make titles stand out ...
(the original title is the much more sedate "More on DOTA 2"; in case it is changed by moderators, at the time of this comment it says "How We Built the Unbeatable DOTA 2 bot", which is not even a quotation from the article)
The submitted title has certainly shaped the commentary (see people reacting to "unbeatable", and whether it's an impressive achievement, vs reacting to the technical content).
So, in a nutshell, I think we should avoid titles like this, and I believe the guidelines transparently say so as well.
[+] [-] sclangdon|8 years ago|reply
So whilst the new edited title is a bit click-baity, it is nevertheless more meaninful than the original.
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] flipgimble|8 years ago|reply
At the same time most of the deep learning field has been focusing on implementing super-human perceptual abilities (vision, hearing, translation) that come instinctually to humans. The higher level reasoning and memory/attention augmented machine learning is still cutting edge research. I think DeepMind and OpenAI are driving research towards that end.
[+] [-] iMarv|8 years ago|reply
[+] [-] ryzawy|8 years ago|reply
[+] [-] doikor|8 years ago|reply
[+] [-] sillysaurus3|8 years ago|reply
My favorite part:
The first step in the project was figuring out how to run Dota 2 in the cloud on a physical GPU. The game gave an obscure error message on GPU cloud instances. But when starting it on Greg’s personal GPU desktop (which is the desktop brought onstage during the show), we noticed that Dota booted when the monitor was plugged in, but gave the same error message when unplugged. So we configured our cloud GPU instances to pretend there was a physical monitor attached.
Dota didn’t support custom dedicated servers at the time, meaning that running scalably and without a GPU was possible only with very slow software rendering. We then created a shim to stub out most OpenGL calls, except the ones needed to boot.
[+] [-] gcp|8 years ago|reply
[+] [-] itchyjunk|8 years ago|reply
So many things don't fit in.. but that might be because I am neither good at DOTA 2 nor at AI stuff. Good luck with the 5v5's openai. And if the AI bots become public, maybe i'll get to play against them some day.
[+] [-] systoll|8 years ago|reply
If you tell the API to make you attack a thing once, you're 'attacking' until the projectile is created, and then you're 'idle' immediately afterward.
So long as you're feeding the hero actions, it'll always animation cancel.
[+] [-] yorwba|8 years ago|reply
What I think happened here: the bot lost against someone using an item that hadn't been whitelisted before. The devs decided to include the item among the possible choices for self-training. Exploring the new choice, the bot adapted to using it to defeat its previous strategy. As a result, it now had to change its strategy to account for an opponent using the item.
[+] [-] murukesh_s|8 years ago|reply
So it's kind of interesting and equally scary that we are looking at a potential future, possibly near, where most of the work can be handled by a trained robot with humans only supervising them in exceptional cases. Robots can then could be only need to be smart enough to alert the human supervisors when they reach a state where they cannot perform the job normally, for e.g. power grid going off in a factory and they couldn't figure out what to do, it could only be a 0.001 % chance. The point is AI need not get as good as humans in learning complex scenarios to replace majority of us, it just need to do 99% of what we repeatedly do, much better than us.
[+] [-] kevinr|8 years ago|reply
[+] [-] simpx|8 years ago|reply
How does the bot understand the value of long-term strategy?
[+] [-] sockmeistr|8 years ago|reply
[+] [-] isseu|8 years ago|reply