[madlag]

I don't think you have the right numbers in mind talking about the compute you need for AI. The prices are getting lower and lower of course, but you still need tons of money to train the kind of networks that make the news.

[wbl]

The resources academia has can be pretty big also. Top 500 doesnt have a lot of corporate machines on it.

[caenorst]

I don't think you have the right numbers in mind talking about the networks used in academic works. The majority of network used in publications are good old references like VGG

[theferalrobot]

Example? I have yet to see something actually deployed by one of the big tech companies that could not be trained by students on a university cluster. I also think you underestimate grant funding. I worked at a state school a couple years back in a research lab that had over a million dollars in grant funds specifically for equipment and outside compute (not for salaries or new hires) and this is not at all abnormal.

[solidasparagus]

State of the art CV models (image, not video) can cost 3-figure dollars per training run.

State of the art language models can cost 5-figure dollars per training run.

There are a lot of variable in play here so your mileage will definitely vary (how much data, how long are you willing to wait, do you really need to train from scratch, etc) and these should only be considered very rough ballpark numbers. However, those are real numbers for SotA models on gold-standard benchmark datasets using cost-optimized cloud ML training resources.

At 5-figures per training run, the list of people who can be innovators in the LM research space is very small (fine-tuning on top of a SotA LM is a different, more affordable matter).