[nrmn]

Not OP but I believe the low score is due to not enough training time and incorrect parameters such as frameskip. Space Invaders was mentioned as one of the few games they needed to lower the frameskip (from 4 to 3 or 2?) on because of the flashing lasers. I'm assuming OP left the parameters as is from the implementation by Nathan Sprague, which has frameskip at 4, and trained for a few epochs.

[alexlikeits1999]

Changing the frameskip is not needed any more since the implementation does max of last two frames before processing (same as what DeepMind does)

EDIT: talking about Sprague's implementation btw, not necessarily OPs.

[mjaskowski]

Actually I did both frame skipping and amxing out two frames as reported in the Natures letter

https://storage.googleapis.com/deepmind-data/assets/papers/D...