[lofatdairy]

Say what you want about Facebook, the size of their dataset and computational resources definitely make them competitive, and their data science and ML teams have been always top notch. I think the Verge is missing the mark with the headline and general focus of the article. "Building AGI" is whatever, like half the companies with enough GPUs are claiming that and AGI is like more poorly-defined than "metaverse". The more interesting point seems to be this general incoherence with building chatbots and trying to run a social media company.

>Meta is still a metaverse company. It’s the biggest social media company in the world. It’s now trying to build AGI. Zuckerberg frames all this around the overarching mission of “building the future of connection.”

This is such "Verge" writing. I'm by no means bearish on VR, but that whole passage is so unreflective and uncritical it's almost a satire of journalistic fluff. Chatbots that fill social media with greater and greater amounts of garbage content is just a nightmare. Bot content is already one of the reasons people are retreating into groupchats. The blurring of AI and human interaction leads to accountability problems. Hell, Snapchat and Discord basically already tried this to enormous backlash. The fact that this is entirely antagonistic with "building the future of connection" goes essentially unacknowledged.

There is something interesting with the fact that Facebook is more open to open-source, this is fairly credible actually given the quality and quantity of the company's open-source contributions. But I genuinely think LLMs are most useful as an applied technology, and the applications listed here are frankly uninspiring.

[maaaaattttt]

Saying they make it “open source” in the same article where they say they need “350k high end GPUs to build it”. Is the equivalent of saying: ”we offer free nuclear submarine driving lessons”.

I know you don’t need as many resources for inference as for training. But still…

[weeblewobble]

What do you mean “but still…”? It’s a pretty important distinction. Meta does indeed use their massive GPU farms to train models and then release the weights for free and people indeed run inference on prosumer hardware

[blackoil]

How different is saying they built React using 100-500+ developer years of effort and then open sourced it. What they are releasing is what is needed by most of the people looking for open models.

[ipsum2]

You can run llama models on a personal computer, even though it was trained on >10,000 GPUs.