[singhrac]

Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question.

A few projects just in ML: DeepMind has plenty enough for Google alone but Jax and the TPU project are technically ambitious and very strategically important for Google; they hired Adam Paszke away from Meta. Besides Gemini they have Gemma and Veo and they have a reputation in the industry of having extremely high MFU averages. From Facebook there's PyTorch but that's a whole cluster of projects (compiler, abstractions for many accelerators, torchao, torchtitan). They're also famous for DINO and SAM models, as well as many others by FAIR (e.g. Mask R-CNN).

[godelski]

It's worth noting that many of those projects were the famous versions of papers, ones that were successful through scale rather than through innovation. I'll give a good example, here's essentially the same architecture as the 16x16 words ViT paper but a year earlier[0]. It's not even the first, they even mention two other works that used transformers on images. I'm all for scaling and it is important, but there's tons of papers like this that are greatly overshadowed because someone just scaled up and got a better benchmark. It's been getting out of hand...

Fuck... I'm starting to sound like Schmidhuber

[0] https://arxiv.org/abs/1911.03584

[high_na_euv]

>Small startups will always excel in talent density because of the increase risk (and reward) and alignment along equity, but all of the large companies have enough technically brilliant people. Many in their 30s or 40s want to do excellent work and also have stability to provide for their families. Whether management gets out of their way is another question.

You mean small startups founded by experienced engineers?