[terramex]

Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).

[hmottestad]

Or Groq. They can do 1250 tokens/s with Llama 3 8B.