[brotchie]

Open question for LLMs, does creativity and new ideas come from a process or is it a laddered emergent capability.

What I mean by this, is the process of coming up with novel ideas a single capability that has to be trained and reinforced.

Or is it a ladder of capabilities of increasing complexity in that a model that could figure of General Relativity from scratch would not be able to continue the process and perhaps come up with a viable “theory of everything.”

One thing I’ve wanted to do, I’m sure somebody has tried it, is build a dataset to RL a model to be more creative: Get a human expert in a field, have them ask a reasoning model some open questions, and then have the expert look at 20 outputs and rank them by creativity / insight. Have the expert iterate and see how much new “insight” they can mine from the model.

Do this across many fields, and then train a model on these rankings.

Perhaps creativity is a different way of moving in latent space which is “ablated” from existing models because they’re tuned to be “correct” rather than “creative.”

Also curious what techniques there are to sample a reasoning model to deliberately perturb its internal state into more creative realms. Though these a fine line between insight and hallucination.

In some respects creativity is hallucination. As a human, you’re effectively internally postulating creative ideas “hallucinations” and then one of them “hits” and fires a whole bunch of neurons which indicate: “ok that wild idea actually has grounding and strong connections to the existing knowledge in your brain.”

[visarga]

I think creativity comes from search, inspiration or discoveries from the environment. Creativity is basically searching and stumbling into novel ideas.

Creativity doesn't come from the brain itself, the brain is just exploring and accumulating experience. Experience builds on itself. The role of the environment is both to spark and to invalidate ideas.

For example AlphaZero with just search-and-learn strategy could beat humans at our own game. It's not a magic of the model, but of the search loop.