[jrimbault]

> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters.

Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?

[vonneumannstan]

>Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?

Right but wrong. Alphago and AlphaZero are built using very different techniques than GPT type LLMs. Google created Transformers which leads much more directly to GPTs, RLHF is the other piece which was basically created inside OpenAI by Paul Cristiano.

[jrimbault]

Yep, I looked it up a few hours after, different branches in the evolution of ML. Still weird to dismiss AlphaZero as just "playing games".

[jimbo808]

Google Brain invented transformers. Granted, none of those people are still at Google. But it was a Google shop that made LLMs broadly useful. OpenAI just took it and ran with it, rushing it to market... acquiring data by any means necessary(!)

[9rx]

> OpenAI just took it and ran with it

As did Google. They had their own language models before and at the same time, but chose different architectures for them which made them less suitable to what the market actually wanted. Contrary to the above claim, OpenAI seemingly "won" because of GPT's design, not so much because of the data (although the data was also necessary).

[ethan_smith]

Agreed - AlphaGo/Zero's reinforcement learning breakthroughs were foundational for modern AI, establishing techniques like self-play and value networks that influenced transformer architecture development.

[msp26]

OpenAI's work on Dota was also very important for funding

[phreeza]

Transformers/Bert yes, alphago not so much.