top | item 46562731

(no title)

Scaevolus | 1 month ago

Are the LLMs run on-device, or does this use cloud compute?

(Off-topic AMA question: Did you see my voxel grid visibility post?)

discuss

order

tom_0|1 month ago

The "big" one is Llama3.3-70b on the cloud, right now. On GroqCloud in fact, but we have a cloud router that gives us several backups if Groq abandoned us.

We use a ton of smaller models (embeddings, vibe checks, TTS, ASR, etc) and if we had enough scale we'll try to run those locally for users that have big enough GPUs.

(You mean the voxel grid visibility from 2014?! I'm sure I did at the time... but I left MC in 2020 so don't even remember my own algorithm right now)