[sho]

Real-world workflows? I'm all for local LLM, tinker with it all the time, but for productive coding use no local LLM approaches cloud and it's not even close. There's no magic trick or combination of pieces, it just turns out that a quarter million dollars worth of H200s is just much, much better than anything a normal person could possibly deploy at home.

Give it time, we'll get there, but not anytime soon.

[exac]

I thought you would just use another computer in your house for the flows?

My development flow takes a lot of RAM (and yes I can run it minimally editing in the terminal with language servers turned off), so I wouldn't consider running the local LLM on the same computer.

[sho]

It's not about which of your computers you run it on, it's about the relative capability of any system you're likely to own vs. what a cloud provider can do. The difference is hilarious - probably 100x. Knowing that, unless you have good reasons (and experimenting/playing around IS a good reason) - not many people would choose to actually base their everyday workflow on an all-local setup.

It's sort of like doing all your work on an 80386. Can it be made to work? Probably. Are you going to learn a whole lot making it work? Without a doubt! Are you going to be the fastest dev on the team? No.

[starik36]

You are right. This is the current situation. Plus the downside is that your laptop heats up like a furnace if you use the local LLM a lot.