[sacred_numbers]

It's unlikely that OSS LLMs will ever be able to compete with corporate LLMs. I can only think of a few scenarios where this could work:

1. Someone develops a procedure for training models with distributed computing resources, including consumer CPUs and GPUs. Even this is not really guaranteed to work, since corporations will probably just buy up all the consumer GPUs, since they are cheaper on a FLOPS/$ metric.

2. One or more governments provide a lot of funding for OSS models, including being willing to pay competitive salaries for the best talent (potentially millions of dollars per year). This is unlikely for a lot of reasons. The only thing that could speed up the process enough to compete with private organizations is a major war that required AI to win. In that case, though, open source would be the least of their concern.

3. Scaling laws stop working and Moore's law catches up. In that case adding more compute won't really help and eventually even organizations with small budgets can afford to train a SOTA LLM. We can't know until we find the limit, but we haven't hit the limit of scaling laws so far, despite scaling up massively over the last few years, so I doubt we will find the limit any time soon.

4. A bunch of corporations that have no hope of reaching first place decide to combine their resources to beat OpenAI and thus prevent a monopoly. I'm not sure if there is precedent, but even if there is, that would still require a lot of coordination and resources for little direct monetary gain.

We'll see what happens, but I'm not really confident about any of these possibilities.