[dauhak]

This makes no sense. You can describe the brain reductively enough and make it sound like it can't have an original insight either. Transformers are expressive enough function approximaters in theory, there's no reason why a future one couldn't have novel insights.

This is such a weird misconception I keep seeing - the fact that the loss function during training is minimising CE/maximizing prob of correct token doesn't mean that it can't do "real" thinking. If circuitry doing "real" thinking is the best solution found by SGD then it obviously will