This is a great solution for a very specific type of team but I think most companies with consistent GPU workloads will still just rent dedicated servers and call it a day.
Other benefits: easy access to reliable infrastructure and latest hardware which you can swap as you please. There are cases where it makes sense to navigate away from the big players (like dropbox going from aws to on-prem), but again you make this move when you want to optimize costs and are not worried about the trade-offs.
I agree, and cloud compute is poised to become even more commoditized in the coming years (gazillion new data centers + AI plateauing + efficiency gains, the writing is on the wall). There’s no way this makes sense for most companies.
The advantage of renting vs. owning is that you can always get the latest gen, and that brings you newer capabilities (i.e. fp8, fp4, etc) and cheaper prices for current_gen-1. But betting on something plateauing when all the signs point towards the exact opposite is not one of the bets i'd make.
ocdtrekkie|26 days ago
Cloud excels for bursty or unpredictable workloads where quickly scaling up and down can save you money.
langarus|26 days ago
hyperbovine|26 days ago
NitpickLawyer|26 days ago
Ummm is that plateauing with us in the room?
The advantage of renting vs. owning is that you can always get the latest gen, and that brings you newer capabilities (i.e. fp8, fp4, etc) and cheaper prices for current_gen-1. But betting on something plateauing when all the signs point towards the exact opposite is not one of the bets i'd make.