[alexilliamson]

Well-defined goal is the big one. We wanted a big bomb.

What does AGI do? AGI is up against a philosophical barrier, not a technical one. We'll continue improving AI's ability to automate and assist human decisions, but how does it become something more? Something more "general"?

[lucubratory]

"General" is every activity a human can do or learn to do. It was coined along with "narrow" to contrast with the then decidedly non-general AI systems. This was generally conceived of as a strict binary - every AI we've made is narrow, whereas humans are general, able to do a wide variety of tasks and do things like transfer learning, and the thinking was that we were missing some grand learning algorithm that would create a protointelligence which would be "general at birth" like a human baby, able to learn anything & everything in theory. An example of an AI system that is considered narrow is a calculator, or a chess engine - these are already superhuman in intelligence, in that they can perform their tasks better than any human ever possibly could, but a calculator or a chess engine is so narrow that it seems absurd to think of asking a calculator for an example of a healthy meal plan, or asking a chess engine to make sense of an expense report, or asking anything to write a memoir. Even in more modern times, with AlexNet we had a very impressive image recognition AI system, but it couldn't calculate large numbers or win a game of chess or write poetry - it was impressive, but still narrow.

With transformers, demonstrated first by LLMs, I think we've shown that the narrow-general divide as a strict binary is the wrong way to think about AI. Instead, LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system. They aren't as good as our existing superhuman computer systems at these tasks (aside from language processing, which they are SOTA at), not even as good at humans, but they're obviously much better than chance. With training to use tools (like calculators and chess engines) you can easily make an AI system with an LLM component that's superhuman in those fields, but there are still things that LLMs cannot do as well as humans, even when using tools, so they are not fully general. One example is making tools for themselves to use - they can do a lot of parts of that work, but I haven't seen an example yet of an LLM actually making a tool for itself that it can then use to solve a problem it otherwise couldn't. This is a subproblem of the larger "LLMs don't have long term memory and long term planning abilities" problem - you can ask an LLM to use python to make a little tool for itself to do one specific task, but it's not yet capable of adding that tool to its general toolset to enhance its general capabilities going forward. It can't write a memoir, or a book that people want to read, because they suck at planning or refining from drafts, and they have limited creativity because they're typically a blank slate in terms of explicit memory before they're asked to write - they have a gargantuan of implicitly remembered things from training, which is where what creativity they do have comes from, but they don't yet have a way to accrue and benefit from experience.

A thought exercise I think is helpful for understanding what the "AGI" benchmark should mean is: can this AI system be a drop-in substitute for a remote worker? As in, any labour that can be accomplished by a remote worker can be performed by it, including learning on the job to do different or new tasks, and including "designing and building AI systems". Such a system would be extremely economically valuable, and I think it should meet the bar of "AGI".

[riffraff]

> LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system

But they can't, they still fail at arithmetic and still fail at counting syllables.

I think that LLMs are really impressive but they are the perfect example of a narrow intelligence.

I think they don't blur the lines between narrow and general, they just show a different dimension of narrowness.