The problem is that training a free and open source model costs just as much as training a closed one, but has even fewer potential avenues for recouping that investment. The money still has to come from somewhere.
I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another.
Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.
I'm not sure about that. Reports have shown that models from China or Mistral can achieve 80% or more of OpenAI's performance for a fraction of the cost.
If you're tucked in right behind the absolute frontier models, the economics change completely
I would laugh my ass off if Coca Cola Company ends up being the company that solves alignment - so that it can align an "open weight" AI with its corporate interests.
Without that though? Our ability to manipulate LLMs is so shaky I would be really surprised if anyone managed to pull off this kind of model manipulation and have it remain undetected.
Just wait until someone leaks an internal SOTA model. Would be deeply ironic given how much AI robber barons ‘respect’ others’ copyright and trade secrets.
That is literally the thing the parent poster wants to avoid by running open models.
[edit] I was a little unfair -- lack of access to training data is a bit of an issue (perhaps moreso for analysis than for for actual use, considering what it takes to train these models). I'm thankful that some of them are also distributed as base models, which should be relatively unbiased compared to what happens later during finetuning.
I’m wondering how long it will be until they are also “sponsored” to have ad content trained in. I personally despise advertising but nobody is building these things out of the goodness of their heart. There needs to be some ongoing incentive to train and release open models.
Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.
jsheard|3 months ago
I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another.
Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.
Adrig|3 months ago
If you're tucked in right behind the absolute frontier models, the economics change completely
ACCount37|3 months ago
Without that though? Our ability to manipulate LLMs is so shaky I would be really surprised if anyone managed to pull off this kind of model manipulation and have it remain undetected.
gldrk|3 months ago
justonceokay|3 months ago
Gracana|3 months ago
[edit] I was a little unfair -- lack of access to training data is a bit of an issue (perhaps moreso for analysis than for for actual use, considering what it takes to train these models). I'm thankful that some of them are also distributed as base models, which should be relatively unbiased compared to what happens later during finetuning.
boppo1|3 months ago
sipjca|3 months ago
the_real_cher|3 months ago
andy99|3 months ago
Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.