[cwyers]

Yeah, there's only two differences between using Markov chains to predict words and LLMs:

* LLMs don't use Markov chains, * LLMs don't predict words.

[arboles]

* Markov chains have been used to predict syllables or letters since the beginning, and an LLMs tokenizer could be used for Markov chains

* The R package markovchain[1] may look like it's using Markov chains, but it's actually using the R programming language, zeros and ones.

[1] https://cran.r-project.org/web/packages/markovchain/index.ht...