I love GTAV so much, it's the most real single player world I've ever experienced. I once rear-ended an NPC because they suddenly stopped in the middle of the street. I got out of my car (because it's GTA and you have to shoot everyone, duh) I saw the NPC had stopped because he hit a cat with his car!
I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML.
Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?
Hmpth. Conversely, while playing GTAV in completely innocent explore-the-world mode, I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head.
Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.
It's technically possible to run everything on one Windows machine (using VNC loopback mode), however we expect most people to run ML on Linux / Mac so that is the best supported configuration. Before Universe, I ran a modified version of Caffe in Windows along with the game on a GTX 980 and there were more than enough GPU resources for that. Running ML / ROS / other self-driving software in Linux is much more ideal though.
I don't remember if GTA V had this same feature as GTA IV, but I remember being impressed in GTA IV by the cell phone interference through the radio when you had an incoming call and the radio was playing. I just absolutely loved the attention to detail that game had. It was truly the small things that added up that made you want to figure out what other details they put in the game.
Ok, I am so glad that we're not training AI to "win" at GTA V. That is not an android I want to meet on the street :-)
The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?
I don't think anyone is thinking of training on GTA then reusing it on a real car. It's just a convenient way to work out the algorithms and architecture in a controlled way that might then work out with the real world data.
But I do think GTA's unrealistic physics engine[1] will hurt it for simulating the cases we care most about (accidents, lost of traction, etc)
Sounds a bit crazy until you look at it from the perspective of being able to visually observe how an AI driver would react to erratic behavior like pedestrians suddenly running onto the road or conditions like gunfire, accidents, etc. I think this is just a small step in the positive direction of AI development.
It's surprising how scriptable the game is. For example, the car I train the AI with is set to the least aggressive driving style which stops for rabbits crossing the road :) Also, all violence against the player is disabled.
It's all fun and games until the Second Amendment is used to protect the ownership of armed drones that can be used to assassinate anyone anonymously simply by scratching the serial numbers off.
ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.
Thanks! Yes, the game is highly scriptable[1], so you can test the types of scenarios you'd want. We've disabled violence against the driver, but could enable[1] it on-demand if it's useful for training.
I'd be super interested in seeing how this reacts to having a single star wanted level. At that level police cars will come to a stop almost perpendicular in front of your car. Will the AI be able to avoid an aggressive obstacle.
Also it'd be interesting to see how it reacts to getting shot at
I wonder if AI is where games can make use of many cores. If the game logic can not be reasonibly parallelized anymore, the unused computing capacity can be used to add "intelligent" agents into the game. They don't need to do revolutionary things, just being more complex, more "reasonable" and less predictable than the usual agents would be enough. I think that has the potential to make some games more interesting.
Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.
Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
> Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.
> Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
It's a really tough trade-off between fun and realism. It's cool that your units have personality for a bit "Oh, look, those 2 dwarves are fleeing from the troll fight early!", but not fun once they disobey your intentions "That fucking troll has 3 hit points left. If my 2 dwarves didn't run away, it would be dead! Now it's destroying my base!"
One game I remember striking a good middle ground with this was the original Battle For Middle Earth. It changed animations, sounds and other aesthetic things based on AI, while the game remained the same mechanically. If your units ran up against a mountain troll, you would see the ones in front of it cowering and holding up shields, while units behind attack furiously, but every unit was really doing the same damage. Or if you left troops of Orcs just standing around too long, they'd break out into fights and kill other orcs, but the animation is done as a trick where as one orc model "dies", a new one spawns in the group. So you didn't really ever lose Orcs, even though there would be orc corpses strewn wherever you had been camping.
I think the main reason though is just that AI doesn't really sell. Graphics, physics and voice actors sell. You can blame a failure on AI, but very rarely is success attributed to good AI in a game.
There's a recent game called Hello Neighbour where the eponymous neighbor learns from the patterns you use to sneak into his house. The one video I saw of the game looked freaky. 10/10 would play.
I don't know about current games (I suspect they have enough to load the CPU with gfx/physics to devote much to AI), but state of the art AI software can definitely be many core. It's very useful for anything doing Monte Carlo simulations, tree search, planning.
Imagine an RTS or something similar where you play with an AI and he learns your behaviours. He tries to help you. At first you deny what he does, he learns, do something else. A bit like the beast in black & white!
I was told (but didn't verify it myself, nor have any sources on this) that in Half Life 2 (2004), your enemies would "learn" from your attack patterns and adapt to it. At the same time, as to make the game enjoyable/chellenging and not frustratingly difficult, the agents would adjust their difficulty level to your skill. Take it as an anecdote.
In ARMA opponent AI is a parameter. When you have a lot of players on one server, you turn down enemy AI so you don't bog down the entire server with pathfinding calculations.
It looks like it's not just using the camera/vnc images as input, but also uses depth data to simulate LIDAR input (http://deepdrive.io/).
I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing.
Open world games seem like a great place to work with AI. What has traditionally prevented more intelligence, in my mind, was the ability of AI to understand and integrate with its environment. For example, if I say I threw a baseball, you can visualize that. In a virtual world, the AI can "see" and potentially interact with every object to increase understanding. Great experiment and I look forward to seeing how far you can take it!
The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit.
Yep, you can tell by the way the helicopter sounds and the walking animation for spiderman. The motion blur also looks very similar. The terrible looking ENB mod makes it difficult to notice, tough.
This is super cool. Are there any good resources/how-to modify GTA V for things like this? I'm looking in to GTA main website and wikies but info for developers looks thin.
Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?
Rockstar's policy is that you're free to mod the single-player game, but any modding whatsoever of the online game is forbidden, no matter how miniscule. They detect it and it's a potential ban.
That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party.
The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.
This feels like an important step in the democratization of AI.
We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.
With all respect I think "easy-to-access" is a bit of an understatement when you are comparing a 3D game to real-world sensor data. They are barely comparable.
too bad GTA V is not open-source for researchers. I don't know how customisable it is, but e.g. experiencing snow/ice or a failure like spontaneous pressure loss in one of the tires would be interesting to simulate.
Yes, it would be great to have source! The modding community does make up for this in some ways. For example, mods provide access to things like finding memory addresses for things like steering angle that weren't available in the API for game scriptors working on GTAV:
Just a heads up to anybody who tries to get this running, you may have to request permission from Amazon to get the required instance going (g2.2xlarge). Apparently they are making sure people aren't trying to set up botnets and bitcoin mining. Just sent in my request, we'll see what happens.
So the training car has a max speed of 10 meters per second or 22mph which the net matches. Future datasets will contain the road's speed limit as well as throttle / brake targets required to reach that speed limit.
How much real-time is needed to get 1k (or 1M) iterations, to improve it so that it moves as fast as the speed limit allows while obeying all the laws?
[+] [-] squeaky-clean|9 years ago|reply
I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML.
Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?
[+] [-] dTal|9 years ago|reply
Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.
[+] [-] cr4zy|9 years ago|reply
[+] [-] slezakattack|9 years ago|reply
[+] [-] ChuckMcM|9 years ago|reply
The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?
[+] [-] semi-extrinsic|9 years ago|reply
Answering standard reading comprehension test questions? - "Why did Pierre meet Natasha at the church?"
[+] [-] empath75|9 years ago|reply
[+] [-] rawnlq|9 years ago|reply
But I do think GTA's unrealistic physics engine[1] will hurt it for simulating the cases we care most about (accidents, lost of traction, etc)
[1] GTA is a regular submission on /r/GamePhysics which collects funny game physics glitches: https://www.reddit.com/r/GamePhysics/search?q=GTA&sort=top&r... https://www.reddit.com/r/GamePhysics/search?q=GTAV&sort=top&... https://www.youtube.com/channel/UCy332GE6lcMVP612IUs7ODw/vid...
[+] [-] midolzzzz|9 years ago|reply
[+] [-] cr4zy|9 years ago|reply
[+] [-] ilya_su|9 years ago|reply
[+] [-] delegate|9 years ago|reply
Next step is obviously teaching it how to kill people in the game and get away from police.
I'd like to see the AI go on a rampage and win :).
That should be enough to wipe off a lot of smiles off our idealistic faces :)
[+] [-] smrtinsert|9 years ago|reply
[+] [-] trento|9 years ago|reply
[+] [-] zxcvvcxz|9 years ago|reply
http://gta.wikia.com/wiki/Pimping
ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.
[+] [-] captainarab|9 years ago|reply
Will be super interesting to see how all of the randomness in GTA V driving influences autonomous driving AIs.
I could see them being a bit more "cautious" in order to deal with all of the crazy drivers (and events) sharing the roads of Los Santos.
[+] [-] cr4zy|9 years ago|reply
[1] http://www.dev-c.com/nativedb/ [2] http://gtaforums.com/topic/822314-guide-driving-styles/
[+] [-] corobo|9 years ago|reply
Also it'd be interesting to see how it reacts to getting shot at
[+] [-] madez|9 years ago|reply
Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.
Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
[+] [-] squeaky-clean|9 years ago|reply
> Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?
It's a really tough trade-off between fun and realism. It's cool that your units have personality for a bit "Oh, look, those 2 dwarves are fleeing from the troll fight early!", but not fun once they disobey your intentions "That fucking troll has 3 hit points left. If my 2 dwarves didn't run away, it would be dead! Now it's destroying my base!"
One game I remember striking a good middle ground with this was the original Battle For Middle Earth. It changed animations, sounds and other aesthetic things based on AI, while the game remained the same mechanically. If your units ran up against a mountain troll, you would see the ones in front of it cowering and holding up shields, while units behind attack furiously, but every unit was really doing the same damage. Or if you left troops of Orcs just standing around too long, they'd break out into fights and kill other orcs, but the animation is done as a trick where as one orc model "dies", a new one spawns in the group. So you didn't really ever lose Orcs, even though there would be orc corpses strewn wherever you had been camping.
I think the main reason though is just that AI doesn't really sell. Graphics, physics and voice actors sell. You can blame a failure on AI, but very rarely is success attributed to good AI in a game.
[+] [-] _John-Titor_|9 years ago|reply
[+] [-] gcp|9 years ago|reply
[+] [-] Le_SDT|9 years ago|reply
[+] [-] JabavuAdams|9 years ago|reply
[+] [-] mtrycz|9 years ago|reply
[+] [-] BickNowstrom|9 years ago|reply
[+] [-] dtromero|9 years ago|reply
[+] [-] forgotmysn|9 years ago|reply
[+] [-] ge96|9 years ago|reply
[+] [-] lxe|9 years ago|reply
I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing.
EDIT: found this: https://web.stanford.edu/class/ee368/Project_Autumn_1516/Rep... on the topic. Is there anything else I should read up on?
EDIT: found lots of articles: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&e...
[+] [-] mysterydip|9 years ago|reply
[+] [-] uptown|9 years ago|reply
https://www.youtube.com/watch?v=T5kooJeXGiI
There's a ton of them on YouTube, but I can't figure out how they make them.
[+] [-] kowdermeister|9 years ago|reply
Some use the GTA V. director: http://kotaku.com/on-youtube-millions-watch-shows-for-childr...
The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit.
80 million views in a month is a lot of money: https://www.youtube.com/watch?v=ipTJNNvW-Gw
Just checked their channel: 4,316,098,805 views
[+] [-] PhilipTrauner|9 years ago|reply
[+] [-] sytelus|9 years ago|reply
Here's few links I found so far:
https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto
http://www.dev-c.com/gtav/scripthookv/
https://www.youtube.com/watch?v=wS8oJTHqf8Q
https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/...
https://steamcommunity.com/sharedfiles/filedetails/?id=55807...
http://gtaforums.com/forum/109-tutorials/
Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?
[+] [-] drivingmenuts|9 years ago|reply
That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party.
The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.
[+] [-] wrsh07|9 years ago|reply
We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.
[+] [-] drcross|9 years ago|reply
[+] [-] celerrimus|9 years ago|reply
[+] [-] dsugarman|9 years ago|reply
This is really awesome, great work to those involved.
[+] [-] iagooar|9 years ago|reply
[+] [-] LeanderK|9 years ago|reply
[+] [-] cr4zy|9 years ago|reply
https://github.com/crosire/scripthookvdotnet/blob/dev_v3/sou...
re snow: It is possible to lay fresh snow anywhere in addition to drive through this area with snow burms, etc..
https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be
Traction is definitely affected in snowy areas.
re tires bursting: it seems possible
https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3...
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] justifier|9 years ago|reply
i'm sure this is to offer relief to the nn but does this further simulate 'reality'?
are our autonomous vehicles seeing at 8 frames per second?
[+] [-] alderz|9 years ago|reply
[+] [-] 9erdelta|9 years ago|reply
[+] [-] hacker_9|9 years ago|reply
[+] [-] cr4zy|9 years ago|reply
[+] [-] ph0rque|9 years ago|reply