This is such devious, but increasingly obvious, narrative crafting by a commercial entity that has proven itself adversarial to an open and decentralized internet / ideas and knowledge economy.
The argument goes as follows:
- The future of AI is open source and decentralized
- We want to win the future of AI instead, become a central leader and player in the collective open-source community (a corporate entity with personhood for which Mark is the human mask/spokesperson)
- So let's call our open-weight models open-source, and benefit from its imago, require all Llama developers to transfer any goodwill to us, and decentralize responsibility and liability, for when our 20 million dollar plus "AI jet engine" Waifu emulator causes harm.
Read the terms of use / contract for Meta AI products. If you deploy it, some producer finds the model spits out copyrighted content, knocks on Meta's door, Meta will point to you for the rest of the court case. If that's the future for AI, then it doesn't really matter whether China wins.
> Read the terms of use / contract for Meta AI products. If you deploy it, some producer finds the model spits out copyrighted content, knocks on Meta's door, Meta will point to you for the rest of the court case. If that's the future for AI, then it doesn't really matter whether China wins.
As much as I hate Facebook, I think that seems pretty… reasonable? These AI tools are just tools. If somebody uses a crayon to violate copyright, the crayon is not to blame, and certainly the crayon company is not, the person using it is.
The fact that Facebook won’t voluntarily take liability for any thing their users’ users’ might do with their software means that software might not be useable in some cases. It is a reason to avoid that software if you have one of those use cases.
But I think if you find some company that says “yes, we’ll be responsible for anything your users do with with our product,” I mean… that seems like a hard promise to take seriously, right?
It's got nothing to do with Meta's social media business directly. Massive as the FB dataset is, it gets mogged by google who, what with their advanced non-PHP-based infra and superior coders, basically have way more and way better and way more accessible data... and their own AI CPUs they made, and a bigger cluster, and faster software, and more store, and so on. Big picture, Google is poised to steamroll Facebook AI-wise, and if no them, then openAI+microsoft
So Meta says "well we will buy tons of compute and try to make it distributed" "we'll make the model open and people will fine-tune with data that they found" and so on. Now google and openAI aren't competing versus meta, they are competing versus meta + all compute owned by amateurs + all data scrapped by all amateurs, which is non-trivial. so it's not so much as aspiring to be #1 as capping the knees of the competition who has superior competitiveness - but people love it because the common man wins here for once.
Anyway, eventually, they'll all be open models. Near future weaker models will run on a PC, bigger models on the cluster, weakest models on the phone... then just weak models on the phone and bigger on the PC.. eventually anything and everything fits on a phone and maybe iWatch. Even Google and openAI will have to run on the PC/phone at this point, it wouldn't make sense not to. Then since people have local access to these devices, it all gets reverse engineered, boom boom boom. now they're all open
It's especially rich coming from Facebook who was all for regulating everyone else in social media after they had already captured the market.
Everyone tries this. Apple tried it with lawsuits and patents, Facebook did it under the guise of privacy, OpenAI will do it under the guise of public safety.
There's almost no case where a private company is going to be able to successfully argue "they shouldn't be allowed but we should" I wonder why so many companies these days try. Just hire better people and win outright.
It has been clear from the beginning that Meta's supposed desire for an open source AI, is just a coping mechanism for the fact that got beat out of the gate. This is an attempt to commoditize AI and reduce OpenAI/Google/Whoever's advantage. It is effective, not doubt, but all this wankery about how noble they are for creating an open-source AI future is just bullshit.
Decentralized inferencing perhaps, but the training is very much centralized around Metas continued willingness to burn obscene amounts of money. The open source community simply can't afford to pick up the torch if Meta stops releasing free models.
Not yet, the bandwidth requirement is too high. But if someone figures this out that's when we will have true open source models. A crowdsourced supercomputer can outcompete any corporation's server farm.
1. Open source is for losers. I'm not calling anyone involved in open source a loser, to be clear. I have deep respect for anyone who volunteers their time for this. I'm saying that when companies push for open source it's because they're losing in the marketplace. Always. No companiy that is winning ever open sources more than a token amount for PR; and
2. Joel Spolsky's now 20+ year old letter [1]:
> Smart companies try to commoditize their products’ complements.
Meta is clearly behind the curve on AI here so they're trying to commoditize it.
There is no moral high ground these companies are operating from. They're not using their vast wisdom to predict the future. They're trying to bring about the future the most helps them. Not just Meta. Every company does this.
It's why you'll never see Meta saying the future of social media is federation, open source and democratization.
Great, who gives me $500,000,000, Nvidia connections to actually get graphics cards and a legal team to protect against copyright lawsuits from the entities whose IP was stolen for training?
Then I can go ahead and train my open source model.
the view of the comments here seems to be quite negative for what meta is doing. Honest question, should they go to the route of openai and closed source + paid access instead? OpenAI or Claude seem to garner more positive views than llama open sourced.
Not much would change if they did. Meta intentions and OpenAI intentions are the same: reach monopoly and take all the investment back with a 100x return. Anyone that achieves it will be as evil as the other one.
> OpenAI or Claude seem to garner more positive views than llama open sourced.
that's more about Meta than the others. Although OpenAI isn't that far from Meta already.
This is the modern form of embrace, extend and extinguish. "Embrace" open source, "extend" the definition to make it non open/libre and finally extinguish the competition by shoring up the drawbridge to the moat they've just built.
It’s marketing to get the best researchers. The researchers want the meta pay and they want to hedge their careers to continue to publish. That’s the real game, it’s a war for talent. Everything else is just secondary effects.
The future of everything you depend on is open source and decentralized.
Because all indications are that the powers over you cannot abide your freedoms of association, communication and commerce.
So, if it’s something your family needs to survive - it has better be distributed and cryptographically secured against interference.
This includes interference in the training dataset of whatever AIs you use; this has become a potent influence on the formation of beliefs, and thus extremely valuable.
All of these models, including the "open" ones, have been RLHF'ed by teams of politically-motivated people to be "safe" after initial foundation training.
"The Future" in the meantime we will keep doing our stuff, building walled gardens of AI generated spam and slop, and claiming our AI models are open source when they are not.
The faster Meta dies, the better it would be.
the reason - i'm a little bearish on AI is due to its cost. small companies won't innovate on models if they don't have billions to burn to train the models.
yet when you look back at history, things that were revolutionary, it was due to low cost of production. web, bicycles, cars, steam engine cars etc.
The first cars, networks, and many other things were not unexpensive. They became so with time and growing adoption.
Cost of compute will continue decreasing and we will reach that point where it is feasible to have AI everywhere. I think with this particular technology we have already reached a no return point
I suspect that models will become smaller, getting pruned to focus on relevant tasks. Someone using an LLM to power tech support chat doesn't want, nor need, the ability to generate random short stories. In this sense, AI is akin to cars prior to assembly line manufacturing: expensive and bespoke machines, with their full potential tapped when they're later made in a more efficient manner.
I could see the cost of licensing data to train models increasing significantly, but the cost of compute for training models is only going to drop on a $/PFLOP basis.
I like the idea of this! But is there any reason to be concerned about walled gardens in this case, like how Apple does with its iOS ecosystem? For example, what if access to model weights could be revoked.
There is a lot of interest in regulating open source AI, but many sources of criticism miss the point that open source AI helps democratize access to technologies. It worries me that Meta is proposing an open source and decentralized future because how does that serve their company? Or is there some hope of creating a captive audience? I hate to be a pessimist or cynic, but just wondering out loud, haha. I am happy to be proven wrong.
Well, yes. They are a company, with shareholders and all. So while not breaching any law, they should indeed pursue strategies that they think would be profitable.
And for all the negativity seen in many of the comments here I think it’s actually quite remarkable that they make model checkpoints available freely. It’s an externality, but a positive one. Not quite there yet in terms of the ideal - which is definitely open source - and surely with an abuse of language, which I also note. But overall, the best that is achievable now I think.
The true question we should be tackling is, is there an incentive-compatible way to develop foundation models in a truly open source way? How to promote these conditions, if they do exist?
Facebook promised to connect the world in a happy circle of friendship and instead causes election integrity controversies, bizarre conspiracy theories about pandemics and immigrants to go viral, and massive increases in teen suicide. Not sure why anyone would trust them with their promises of decentralized-AI and roses.
> In 2012, Red Hat Inc. accused VMWare Inc. and Microsoft Corp. of openwashing in relation to their cloud products.[6] Red Hat claimed that VMWare and Microsoft were marketing their cloud products as open source, despite charging fees per machine using the cloud products.
Other companies are way more careful using "open source" in relation to their AI models. Meta now practically owns the term "Open Source AI" for whatever they take it to mean, might as well call it Meta AI and be done with it: https://opensource.org/blog/metas-llama-2-license-is-not-ope...
rkou|1 year ago
This is such devious, but increasingly obvious, narrative crafting by a commercial entity that has proven itself adversarial to an open and decentralized internet / ideas and knowledge economy.
The argument goes as follows:
- The future of AI is open source and decentralized
- We want to win the future of AI instead, become a central leader and player in the collective open-source community (a corporate entity with personhood for which Mark is the human mask/spokesperson)
- So let's call our open-weight models open-source, and benefit from its imago, require all Llama developers to transfer any goodwill to us, and decentralize responsibility and liability, for when our 20 million dollar plus "AI jet engine" Waifu emulator causes harm.
Read the terms of use / contract for Meta AI products. If you deploy it, some producer finds the model spits out copyrighted content, knocks on Meta's door, Meta will point to you for the rest of the court case. If that's the future for AI, then it doesn't really matter whether China wins.
bee_rider|1 year ago
As much as I hate Facebook, I think that seems pretty… reasonable? These AI tools are just tools. If somebody uses a crayon to violate copyright, the crayon is not to blame, and certainly the crayon company is not, the person using it is.
The fact that Facebook won’t voluntarily take liability for any thing their users’ users’ might do with their software means that software might not be useable in some cases. It is a reason to avoid that software if you have one of those use cases.
But I think if you find some company that says “yes, we’ll be responsible for anything your users do with with our product,” I mean… that seems like a hard promise to take seriously, right?
Calvin02|1 year ago
amy-petrik-214|1 year ago
So Meta says "well we will buy tons of compute and try to make it distributed" "we'll make the model open and people will fine-tune with data that they found" and so on. Now google and openAI aren't competing versus meta, they are competing versus meta + all compute owned by amateurs + all data scrapped by all amateurs, which is non-trivial. so it's not so much as aspiring to be #1 as capping the knees of the competition who has superior competitiveness - but people love it because the common man wins here for once.
Anyway, eventually, they'll all be open models. Near future weaker models will run on a PC, bigger models on the cluster, weakest models on the phone... then just weak models on the phone and bigger on the PC.. eventually anything and everything fits on a phone and maybe iWatch. Even Google and openAI will have to run on the PC/phone at this point, it wouldn't make sense not to. Then since people have local access to these devices, it all gets reverse engineered, boom boom boom. now they're all open
unknown|1 year ago
[deleted]
eli_gottlieb|1 year ago
jongjong|1 year ago
bschmidt1|1 year ago
Everyone tries this. Apple tried it with lawsuits and patents, Facebook did it under the guise of privacy, OpenAI will do it under the guise of public safety.
There's almost no case where a private company is going to be able to successfully argue "they shouldn't be allowed but we should" I wonder why so many companies these days try. Just hire better people and win outright.
ilrwbwrkhv|1 year ago
[deleted]
foobar_______|1 year ago
jsheard|1 year ago
leetharris|1 year ago
The #1 problem is not compute, but data and the manpower required to clean that data up.
The main thing you can do is support companies and groups who are releasing open source models. They are usually using their own data.
unknown|1 year ago
[deleted]
cynicalpeace|1 year ago
AFAICT it decentralizes the training of these models by giving you an incentive to train models which will mine the crypto if you're improving it.
I learned about it years ago, mined some crypto, lost the keys and now kicking myself cuz I would've made a pretty penny lol
numpad0|1 year ago
monkeydust|1 year ago
WithinReason|1 year ago
jmyeet|1 year ago
1. Open source is for losers. I'm not calling anyone involved in open source a loser, to be clear. I have deep respect for anyone who volunteers their time for this. I'm saying that when companies push for open source it's because they're losing in the marketplace. Always. No companiy that is winning ever open sources more than a token amount for PR; and
2. Joel Spolsky's now 20+ year old letter [1]:
> Smart companies try to commoditize their products’ complements.
Meta is clearly behind the curve on AI here so they're trying to commoditize it.
There is no moral high ground these companies are operating from. They're not using their vast wisdom to predict the future. They're trying to bring about the future the most helps them. Not just Meta. Every company does this.
It's why you'll never see Meta saying the future of social media is federation, open source and democratization.
[1]: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
Qshdg|1 year ago
Then I can go ahead and train my open source model.
unknown|1 year ago
[deleted]
riku_iki|1 year ago
hyuuu|1 year ago
naming_the_user|1 year ago
meiraleal|1 year ago
> OpenAI or Claude seem to garner more positive views than llama open sourced.
that's more about Meta than the others. Although OpenAI isn't that far from Meta already.
troupo|1 year ago
Now ask yourself a question: where does Meta's data come from? Perhaps from their users' data? And they opted everyone in by default. And made the opt-out process as cumbersome as possible: https://threadreaderapp.com/thread/1794863603964891567.html And now complain that the EU is preventing them from "collecting rich cultural context" or something https://x.com/nickclegg/status/1834594456689066225
VeejayRampay|1 year ago
abetusk|1 year ago
uptownfunk|1 year ago
pjkundert|1 year ago
Because all indications are that the powers over you cannot abide your freedoms of association, communication and commerce.
So, if it’s something your family needs to survive - it has better be distributed and cryptographically secured against interference.
This includes interference in the training dataset of whatever AIs you use; this has become a potent influence on the formation of beliefs, and thus extremely valuable.
caeril|1 year ago
All of these models, including the "open" ones, have been RLHF'ed by teams of politically-motivated people to be "safe" after initial foundation training.
unknown|1 year ago
[deleted]
alecco|1 year ago
* does not apply to training data
lccerina|1 year ago
stonethrowaway|1 year ago
This is chess pieces being moved around the board at the moment.
unknown|1 year ago
[deleted]
dzonga|1 year ago
yet when you look back at history, things that were revolutionary, it was due to low cost of production. web, bicycles, cars, steam engine cars etc.
rafaelmn|1 year ago
Nuclear everything, rockets/satellites, tons of revolutionary things that are very expensive to produce and develop.
Also software scales differently.
miguelaeh|1 year ago
Cost of compute will continue decreasing and we will reach that point where it is feasible to have AI everywhere. I think with this particular technology we have already reached a no return point
Manuel_D|1 year ago
farco12|1 year ago
zwijnsberg|1 year ago
menacingly|1 year ago
latchkey|1 year ago
candiddevmike|1 year ago
nis0s|1 year ago
There is a lot of interest in regulating open source AI, but many sources of criticism miss the point that open source AI helps democratize access to technologies. It worries me that Meta is proposing an open source and decentralized future because how does that serve their company? Or is there some hope of creating a captive audience? I hate to be a pessimist or cynic, but just wondering out loud, haha. I am happy to be proven wrong.
croes|1 year ago
Refusing23|1 year ago
If that means it'll be free/cheaper... sure
Marcus_Ford|1 year ago
[deleted]
unknown|1 year ago
[deleted]
exabrial|1 year ago
dkga|1 year ago
And for all the negativity seen in many of the comments here I think it’s actually quite remarkable that they make model checkpoints available freely. It’s an externality, but a positive one. Not quite there yet in terms of the ideal - which is definitely open source - and surely with an abuse of language, which I also note. But overall, the best that is achievable now I think.
The true question we should be tackling is, is there an incentive-compatible way to develop foundation models in a truly open source way? How to promote these conditions, if they do exist?
CatWChainsaw|1 year ago
atq2119|1 year ago
mrkramer|1 year ago
pie420|1 year ago
This must be a sign that Meta is not confident in their AI offerings.
troupo|1 year ago
They use "open source" to whitewash their image.
Now ask yourself a question: where does Meta's data come from? Perhaps from their users' data? And they opted everyone in by default. And made the opt-out process as cumbersome as possible: https://threadreaderapp.com/thread/1794863603964891567.html And now complain that the EU is preventing them from "collecting rich cultural context" or something https://x.com/nickclegg/status/1834594456689066225
rkou|1 year ago
> In 2012, Red Hat Inc. accused VMWare Inc. and Microsoft Corp. of openwashing in relation to their cloud products.[6] Red Hat claimed that VMWare and Microsoft were marketing their cloud products as open source, despite charging fees per machine using the cloud products.
Other companies are way more careful using "open source" in relation to their AI models. Meta now practically owns the term "Open Source AI" for whatever they take it to mean, might as well call it Meta AI and be done with it: https://opensource.org/blog/metas-llama-2-license-is-not-ope...