top | item 36659576

(no title)

anandvc | 2 years ago

Can anyone help me understand this: How does the AI "see" what's happening on the screen? Is there some sort of object/scene recognition going on?

discuss

order

cfiggers|2 years ago

It looks like the `dist` function is directly accessing key game values by reading bytes out of the emulator's memory in order to "see" the current game state. This is explained a bit under the "Distance" section.

capableweb|2 years ago

Quick scan of the code (https://github.com/phronmophobic/clj-libretro/blob/67f186e87...) seems to indicate that there is at least some screenshotting going on, + serialization of game state. What is used for detecting thing, I couldn't find from just skimming the code.

sbierwagen|2 years ago

I think the screenshots are just for assembling the time lapse video.

Check out `(defn dist` or read the explanation of that function in the blog post. It can't see enemies or pickups at all. It can only tell how far away it is from the end of the level, and if it's dead or not, then brute forces every button combination for every frame. The "in progress" video shows that it spends most of its time falling into pits, but eventually makes it to the end.

A fun metric would be how many deaths it takes to complete a given level. It's got to be in the tens of thousands.