[albertzeyer]

v0: 16M Parameters

v0.5 123M Parameters

v1: 700M Parameters

v2mini-eval1: 300M Parameters

I would not call this LLM. This is not large. It's just a normal-sized LM. Or even small.

(It's also not a small LLM.)

[efreak]

GPT2 at 774m is considered a LLM. I wouldn't say there's much difference between that and 700m, or even 123M.

Having said that, looking up small language model these days returns tons of results calling 7B models small language models.

------

My understanding of small language models is that they're generally intended for specific purposes, like analysis and classification (whatever you'd call the text equivalent of image interrogation with clip models), translation, etc; that there small because they don't need to be big to do their intended functions, not because they're just smaller versions of bigger models.