Local models are quite capable. Obviously a 4B model isn't going to do the job of a trillion parameter SOTA model but there are many local models that are both fast and very usable for these agentic flows.
Qwen 30B and GLM Flash (also around 30B) are both very good for example and I use them regularly.
No comments yet.