[captainclam]

Crucially, this doesn't just require noise but it requires "taste."

I tend to fall back on music creation as an example of this notion. Lots of innovation in music is experimentation/exploration of "noise," (not necessarily literal white noise) but requires the ear of a discerning musician who ultimately goes "Ooh! I liked that" or passes a "generated sample" by.

This is where I wonder if LLMs can ever innovate. I'm not sure they can develop "taste" for things outside of their distribution. However, I could just as easily be convinced that humans can't either, and sophisticated "taste" is just the exploration of obscure regions of the combinatorial space generated from previously observed samples!