[cangeroo]

What's different?

I imagine that it's increasingly hard to contribute or even participate in the ongoing research, which is dominated by major companies?

I work in a different field, but the way of working now is so different from when I started. It feels like factory work / hamsterwheeling. It used to feel more like the work of an architect, a mixture of art and engineering.

[xanderlewis]

Probably the fact that even ten years ago one could hope to do something impressive (and maybe even state of the art) with a custom-built model running on a laptop... these days anything you might try is likely to be outdone by a more general model.

Like the fact that all sorts of fun but elementary language processing tasks that previously required some clever engineering or at least careful data collection and a custom-built model can just be done (and done much better) by something like GPT-x. There's far less incentive to play around with basic ML stuff as an amateur now.