[nshm]

No, there are mathematical reasons LLMs are better. They are trained with multiobjective loss (coding skills, translation skills, etc) so they understand the world much better than MLM. Original post discuss that but with more words and points than necessary.

[visarga]

GPTs also get gradients from all tokens, BERT only on 15% masked tokens. GPTs are more effective.

[Der_Einzige]

Call it a CLM vs MLM, not LLM vs MLM. Soon LMLM's will exist, which will be LLMs too...