Google/Rulesofthumb

[the_duke]

There is nothing new or dystopian about this, a tradeoff between capital expenditure and labour has existed since ... forever.

Do I buy expensive tool X or get buy with cheap ones that make the work take longer or be less precise? Do I buy fancy machine Y or pay 10 people to do it manually?

The only new development with AI is that this was traditionally limited to relatively manual or repetitive processes, but is now expanding to knowledge work as well.

In the medium term the question will be do I just pay for people, or do I spend resources on collecting data and training a model?

[novagameco]

Dystopia is when white collar workers are no longer secure in the labor force. Basic economics is when blue collar workers are no longer secure in the labor force

[j2kun]

> costs of fixed assets

Minor quibble unrelated to the main content of the post: the measures are not fixed costs of assets, but a blend of depreciation and the operating cost of power usage for those assets. Sort of a regular snapshot of an average daily accounting cost, so to speak (which is reasonable). And this was cost to Google, not taking into account what Google could make by charging Cloud customers for the use of those assets (opportunity cost).

My understanding is that the main use of this tool was actually for engineers to give reasonable-ish impact statements for their performance work. I hadn't heard of anyone using it to make serious trade-offs in project planning, since at the level where that matters, the capacity planning teams had more precise costs related to their actual budgets, as well as short term goals like "RAM has a supply chain shock so we can't get any more than X for the next quarter."

[progbits]

Yup that's right.

Also a pet peeve of mine, people constantly screwed up the units. "10 SWEs" is rate (cost per time), same as "10 TB of RAM", but "SWE-years" is cost (ie dollars). Many design documents use these inconsistently.

[Jabbles]

> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit.

But the exact numbers are important. An H100 is ~$2/hour [1], so 1000 is $16M/year (24/7). Even if Google gets a massive internal discount that's still way more than 3 people's total cost. If you have to choose between 16 (highly paid, senior) people or 1000 H100s would you have to think about the choice?

Then again when you revisit this comment in a few years' time the original comparison may be correct.

[1] https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-p...

[ExoticPearTree]

I don’t think the author meant for this to be treated as an absolute, more like food for thought.

And if you really insist on 24/7 comparison, you would come to need about 12 people, as the expected productivity per person is of about 6 hrs/day. Factor in the fact that people need vacations, sick days, weekends off… it looks like 1000 H100 might actually be a good trade-off in the very near future.

[silvestrov]

purchase price != sales price.

Very unlikely that Google pays $16M for a single H100. Amazon has it at $44k: https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphic...

[setheron]

It also treats SWEs as identical; I wish they were a commodity like hardware....

Unfortunately 1 Jeff Dean SWE != SWE

[insecteblond14]

I wish this internal website existed at my company. This would help throw crazy requests from the business out the window more easily

[bananapub]

make a trivial spreadsheet to do it then, or fifty lines of javascript. it's an excellent tool for sharpening thinking early on, as long as people don't come to think of it as anything more than a rule of thumb.

[phyrex]

What keeps you from writing it up?