[fusionadvocate]

What is holding back AI is this business necessity that models must perform everything. Nobody can push for a smaller model that learns a few simple tasks and then build upon that, similar to the best known intelligent machine: the human.

If these corporations had to build a car they would make the largest possible engine, because "MORE ENGINE MORE SPEED", just like they think that bigger models means bigger intelligence, but forget to add steering, or even a chassi.

[dehugger]

I agree. I want to be able to get smaller models which are complete, contained, products which we can run on-prem for our organization.

I'll take a model specialized in web scraping. Give me one trained on generating report and documentation templates (I'd commit felonies for one which could spit out a near-conplete report for SSRS).

Models trained for specific helpdesk tasks ("install a printer", "grant this user access to these services with this permission level").

A model for analyzing network traffic and identifying specific patterns.

None of these things should require titanic models nearing trillions of parameters.

[furyofantares]

This is extremely theorycrafted but I see this as an excellent thing driving AI forward, not holding it back.

I suspect a large part of the reason we've had many decades of exponential improvements in compute is the general purpose nature of computers. It's a narrow set of technologies that are universally applicable and each time they get better/cheaper they find more demand, so we've put an exponentially increasing amount of economical force behind it to match. There needed to be "plenty of room at the bottom" in terms of physics and plenty of room at the top in terms of software eating the world, but if we'd built special purpose hardware for each application I don't think we'd have seen such incredible sustained growth.

I see neural networks and even LLMs as being potentially similar. They're general purpose, a small set of technologies that are broadly applicable and, as long as we can keep making them better/faster/cheaper, they will find more demand, and so benefit from concentrated economic investment.

[fnord123]

They aren't arguing against LLMs They are arguing against their toaster's LLM to make the perfect toast from being trained on the tax policies of the Chang Dynasty.

[cruffle_duffle]

That’s just machine learning though!