[napo]

It sounds nice (maybe too nice?). I always wanted to see that it would be necessary to have a "sleeping" phase in AI.

It always felt weird that we have to sleep, it doesn't seem to give any evolutionary advantages.

[richardw]

I have an unbaked theory, but the very short version is:

- Animals that have peaks of energy use outcompete animals that have a steady-state energy use. Catch the animal, then rest and recover. For any given amount of energy, this means we can recruit more in a smaller window compared to an animal that plods along with no recuperative phase.

- Many things happen when you're sleeping. Rather than having everything running 24/7, having different phases means we can specialise action and recovery. Since the time is already driven by energy demands, many parts of our body and mind leverage it for different purposes.

[LesZedCB]

1 day of in-context learning and 1 night of fine-tuning on context. that's my pet theory, just shooting from the hip as a total layperson.

[roomey]

It must give an evolutionary advantage, or we wouldn't sleep.

It may be hard to pin point exactly what advantage, but as we do it, it must have given us an advantage!

[dougmwne]

Especially considering that it is so widespread in nearly every creature with a brain. And it’s not simply a period of motionless energy conservation but has very specific neural patterns. The science is definitely zeroing in on a connection to learning.

[nomel]

You've, intentionally or not, ignored a whole body of research. An extremely cursory dive into sleep will show all sorts of functional reasons, related to memory formation.

[visarga]

Continual learning. When models will do that, they will have to sleep as well to avoid catastrophic forgetting.