[nvrmnd]

I don't think that's accurate, it generates novel outputs that were not observed in the training data.

[thomastjeffery]

It doesn't generate new tokens.

Train an LLM on text that only uses lowercase, and it will never output an uppercase letter.

[nvrmnd]

So the model is limited to using words and characters that already exist. I agree with you but I don't see why is a limitation worth pointing out.

[pixl97]

But can it put two tokens together

10 01 = 1001?