[rrsp]

We’re 5+ years from the transformer and we’re still using the transformer for the most cutting edge llms. I don’t see what difference another 5 year is going to make unless someone invents something new that can surpass the transformer, and given the amount of money and resources that has been put into AI since 2017 and the lack of innovation since (in terms of fundamental architecture, not things like Lora and Rope) then I’d say the chances are way way lower than 50%.

[famouswaffles]

Who says you need anything other than the transformer? We've clearly not squeezed all the capability out of it.

[rrsp]

I don’t think it’s so clear. The transformer has been available for 6 years, if it were possible to train one to achieve AGI then what’s stopped anyone from doing this that won’t still be the case in 5 years time, given than there’s potentially ?trillions on the table for anyone that does.