[yshui]

Any autoregressive model can do what you are describing. transformers are generating one token at a time too, not all at once.

[intalentive]

True but memory requirements grow with sequence length. For recurrent models the memory requirement is constant. This is why I qualified with "low memory".

[whimsicalism]

yes but transformers are much slower than state space models