[scottjad]

Starcraft is a game that is played with mouse and keyboard (or game controller on console). I think in order to say an AI has beat a human, it should have to use the same interface that a human uses. That's part of the definition of the game.

In Go, anyone can place a stone as well as anyone else, it's deciding where to place a stone that's the game. In Starcraft, not everyone can control the interface to the game equally well. Playing Starcraft without using the same interface as humans is like playing baseball with something other than a baseball bat.

Also, I think it's unfortunate that they're choosing Starcraft 1: Brood War (presumably because of API reasons) instead of Starcraft 2, since there are many more top-level players playing sc2 right now to give it a run for its money, with an evolving meta that could more likely adapt and challenge the AI after it wins some matches. SC2 is where the tournaments and money is right now (compared to BW), hence the best players devoting the most time.

[barbs]

From the 2015 AIIDE Starcraft AI competition report[0].

> ""Why not StarCraft 2?"

> This is the question we always get asked when we tell people we are doing a BroodWar AI competition. This competition relies completely on BWAPI as a programming interface to BroodWar. BWAPI was created by reverse engineering BroodWar and relies on reading and writing to the program memory space of BroodWar in order to read data and issue commands to the game. Since any program that does this can essentially be seen as a map hack or cheat engine, Blizzard has told us that they don't want us to do anything similar for StarCraft 2. In fact, most of the StarCraft 2 EULA specifically deals with not modifying the program in any way. We are happy that Blizzard have allowed us to continue holding tournaments using BWAPI, and they have also helped out by providing prizes to the AIIDE tournament, however until their policy changes we will not be able to do the same for StarCraft 2."

[0] http://webdocs.cs.ualberta.ca/~cdavid/starcraftaicomp/report...

[beambot]

So why not use Matt's version for SC2 that doesn't provide map hacks?

https://graphics.stanford.edu/~mdfisher/GameAIs.html

[derefr]

It has always seemed to me that there is a truly "proper" way to construct an AI 'bot' for a networked game, and that is to write your own headless game client software that connects to the game's network—emulating the physics well enough to avoid desynchronization with one's opponent, but otherwise being entirely its own program, rather than having anything to do with the reference client per se.

[stared]

Is it problematic to capture screen and generate mouse/keyboard input?

In that way there would be no need of any privileged API... and it would be more natural (i.e. interacting with the real way, instead of its abstracted version).

[rasz_pl]

writing API library based on screen capping is what, 100? 200 man hours job?

[rcheu]

BW has more than enough players to challenge the AI, there's still plenty of players like Bisu, Flash, Effort, NaDa, etc. There's going to be multiple BW tournaments broadcast this year too.

I think there's no need to make it use a keyboard/mouse, you should just add in some human like constraints that are assumed in the game (some limit on APM, some delay between moving the screen and being able to take an action/observe the state).

The interesting part would be whether they can build an AI, not a robot. It's unfortunate that both games are pretty figured out at this point though, it'd be interesting to see if an AI could be made that could react to a big metagame change like http://wiki.teamliquid.net/starcraft/Bisu_Build

[ajuc]

Another thing favouring sc2 is - micro is less important in sc2 than in sc1 (bw).

Still, if there's no api then there's no point complaining.

[kriro]

I think APM should simply be passed as a parameter. Max it out at whatever is the theoretical maximum for a human (how often can the fastest person click buttons). Then you can challenge a player to a game where you offer something like 10% less than their average APM over the last n games. It would actually be quite interesting to see how much APM actually influences the victory. You can easily test this by having cloned AIs play themselves at different APM values and come up with an added value per added APM measure.

[ludamad]

But Brood War has been thoroughly explored, and is a much more stable target (SC2 is prone to game-shaking patches). Back when pro play was just ending, I was astonished by the level of play. While players may need a bit to practice against the AI, a good AI spurring more play is really exciting to me.

[levemi]

> (SC2 is prone to game-shaking patches)

It's something humans have to put up with, the AI should too?

[nkrisc]

By forcing a bot to use a keyboard and mouse, it's no longer so much an AI challenge but an engineering challenge of creating robotic limbs that match human performance, totally unrelated. The two aspects should be separated. Arguably, the AI aspect is far more interesting as it relates to StarCraft.

[obl]

I think the point was mostly to only let the bot have as much information as a player would get on the screen : no omniscient knowledge about the state of every visible unit on the map. You'd probably also want to rate limit the inputs, including changing the viewport position, to something reasonable.

[SerLava]

> it should have to use the same interface that a human uses.

Yeah, I've followed Starcraft AI automation for a while. The people programming these AI systems have basically been creating individual unit AIs with a tiny bit of latency.

It's cool to watch but absolutely does not advance the game theory of Starcraft. If you actually had to ration moves, they would have to come up with quite interesting systems. I'd be fine with something borderline impossible like 500 APM - as fast as you can physically click buttons with 10 fingers. Not 2000 APM.

[hiq]

Limiting the number of APM could be a solution to the bias you mention, especially if the AI eventually manages to beat an expert human player with lower average APM. Of course having a real robot using a mouse and a keyboard would be even better in the long run.

[erm]

SC:BW is quite different than SC2. It would be much more interesting to see this on SC:BW, regardless of where the money/tournaments currently exist.

[narsil]

Agreed. The DeepMind team have already done this with other Atari games: https://youtu.be/08Cl7ii6viY?t=11m35s (Oct 2015)

"The system only gets raw pixels as input."

The results were pretty great, so it would be fascinating to see this work with Matt's version of SC2 as mentioned elsewhere in this thread: https://news.ycombinator.com/item?id=11326119

[daveguy]

Raw pixels. And the score. The score was separate, or rather a signal representing increased score. Relevant quote from the paper:

"The emulator’s internal state is not observed by the agent; instead it observes an image xt ∈ Rd from the emulator, which is a vector of raw pixel values representing the current screen. In addition it receives a reward rt representing the change in game score."

The paper: http://arxiv.org/abs/1312.5602

[foota]

I imagine this would be massively more difficult to do with SC2, due to the much higher variation in pixels.

[JabavuAdams]

> Starcraft is a game that is played with mouse and keyboard (or game controller on console). I think in order to say an AI has beat a human, it should have to use the same interface that a human uses. That's part of the definition of the game.

That's the easy part, though. If you can make a SC AI play well against a human you can certainly send mouse events into a window. It really doesn't show anything.

See for example Sikuli, which you can use for automated testing.

[shultays]

AI is still giving same commands as a player, move unit to there, attack that unit. Even if it was limited to do one action per frame, assuming 60 fps, that would be 3600 action per minute and still x15-x20 times of a good sc player.

Further limiting the apm would be no different than limiting the AI itself. It wouldn't be any different than limiting the cpu/memory usage imo.