jeffclune | 8 years ago | on: Surprising Creativity: Anecdotes from Evolutionary Computation
jeffclune's comments
jeffclune | 8 years ago | on: Genetic Algorithms for Training Deep Neural Networks for Reinforcement Learning
Hello all. I am one of the paper's coauthors. Thank you for your interest in this work! We hope you enjoy it. Just an FYI, this paper is part of a cluster of five papers released on Tuesday morning, detailed in this blog post: https://eng.uber.com/deep-neuroevolution
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
No, it does not. See my reply to similar comments.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
I would just unleash evolution. 1 or 2 in the first generation is a toehold, and from there evolution can begin to do its work. You can also try a larger population (e.g. 2000) and let it run for a while.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
> Likewise, I was not very surprised that you can produce fooling images, but it is surprising and concerning that they generalize across models. It seems that there are entire, huge fooling subspaces of the input space, not just fooling images as points. And that these subspaces overlap a lot from one net to another,
Agreed. That is surprising, and also increases the security risks, because I can produce images on my in-house network and then take them out into the world to fool other networks without even having access to the outputs of those networks.
> likely since they share similar training data (?) unclear.
The original Szegedy et al. paper shows that these sort of examples generalize even to networks trained on different subsets of the data (and with different architectures).
> Anyway, really cool work :)
Thanks. :-)
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
See above. These images were accepted to a juried art competition and posted in a museum:
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
Here they are: http://www.evolvingai.org/fooling
There is also a video summary of the paper there.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
The paper does not use the weights of the network to produce (nearly all of) these images (see the responses to Karpathy above).
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
In fact, we submitted them to an art competition and they were accepted and put up in a museum!
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
Also, you don't necessarily need a lot of predictions from a net to fool it, because (another surprising result) images that fool one net tend to fool others! So I can create fooling images on my in-house net and then take them and fool your net-used-for-some-important-application without getting any feedback from your net. That's very surprising, and does raise serious security concerns.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
You say that these images are highly optimized to produce this effect and would not occur by chance, but have you looked at the images in the "fooling" paper?
http://www.evolvingai.org/fooling
Some of them are very simple, and DO occur a lot in the world. For example, the alternating yellow and black line pattern would be encountered by a driverless car, and it would think it is seeing a school bus.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
Surprisingly, ensembles do not really help. We have tried this and it does not work (the final paper for CVPR 2015 will show these results). Also see the work of Szegedy et al. and Goodfellow et all, which also show that ensembles do not really help.
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
Ha. Jason and I were typing at the same time. :-)
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
They would look like optical illusions!
jeffclune | 11 years ago | on: Images that fool computer vision raise security concerns
Hi Andrej,
I agree with most of what you say, but note that nearly all of the images in the paper were generated without the gradient. I.e. all the images produced by evolution did not use the gradient, only the output of the network regarding its prediction confidence. There are some images that use the gradient, but only to show a 3rd class of "fooling images".
PS. It's nice to see our work (both this paper and the NIPS paper on transfer learning) in your class. Thanks for including it. I wish I could have my students take your course!
jeffclune | 14 years ago | on: Breed 3-D Printable Objects, No Technical Know-How Needed
Sorry. The site is getting hammered right now due to the press articles and Hacker News. We just did a bit of hacking and hopefully it will work now!
jeffclune | 14 years ago | on: Breed 3-D Printable Objects, No Technical Know-How Needed
No. People just tend to choose ones that are.
Here are some that are mostly bilaterally symmetrical, but with left-right asymmetry.
http://endlessforms.com/o/69562/ (ghost) http://endlessforms.com/o/60967/ (mad hatter) http://endlessforms.com/o/39225/ (Dr. Evil)
There are some with no symmetry at all....
jeffclune | 14 years ago | on: Breed 3-D Printable Objects, No Technical Know-How Needed
Here is a longer article, with pictures of evolved objects printed in Silver and Bronze:
http://www.physorg.com/news/2011-08-endless-web-d-printable....
jeffclune | 14 years ago | on: Breed 3-D Printable Objects, No Technical Know-How Needed
That's because you know how to use Blender, which most people don't. But the site is not for a situation when you have a specific design in mind, instead it is for when you want to explore the space of possible designs and see suggested designs that you might not think of on your own, such as this: http://endlessforms.com/o/372201/
But I also doubt I could produce all of these shapes in Blender. I would have no idea how to do these, for example:
http://endlessforms.com/o/103181/ http://endlessforms.com/o/82117/
Best, Jeff
jeffclune | 14 years ago | on: AT&T's Rube Golbergian Web Form
I noticed a surge in traffic to EndlessForms.com and realized it was coming from this post. Thanks to Joel for mentioning our site[1]! For those of you that don't know what I am talking about, see Joel's comment on the original post where he recommends EndlessForms.com, where you can design objects with evolution and 3D print them. Thanks all for checking it out.
page 1
Here are two fun gifs of clever solutions AI came up with:
https://twitter.com/jeffclune/status/974718199722795008
https://twitter.com/jeffclune/status/973605950266331138
Here are some press articles which provide a shorter summary of some fun anecdotes.
popularmechanics.com/technology/robots/a19445627/the-hilarious-and-terrifying-ways-algorithms-have-outsmarted-their-creators/
https://www.newscientist.com/article/8-8-hilarious-ways-ai-h... (paywalled unfortunately)