[nwiswell]

Once physical hardware is involved, the problem gets a lot more complicated.

If you're talking about $2M worth of 4090s you have to concern yourself not only with putting them in hosts, but also with adequate space, power distribution, cooling, and networking. You have to figure out how to gracefully handle hardware failures. You have to install operating systems, or at least come up with some mechanism to deploy automatically. And you just straight up have to supply labor to get 1000+ of these things plugged in.

All that before you can get even get to the "fun part". If you're a multi-millionaire you might be willing to just pay the spot price premium.

[NavinF]

Naw the dude had a year. It's not that hard to pay 1 contractor $30/hr to set up PXE boot, assemble machines and GPUs, air handling units, GPUs, and electrical sub panels. I've done a similar "jack of all trades" job for a weed company in Oakland way back before I became a software engineer. 1000 GPUs only takes 2 weeks to deploy.