I don't know, but I'd posit that anybody who answers "definitely yes" or "definitely no" doesn't know what they are talking about. And that's because we, collectively, really have no way of knowing how close to AGI we are. We struggle enough to define and measure human intelligence, and adding what is arguably a new "kind" of intelligence to the pot just complicates matters even more.
So as of today, we don't really have any kind of simple metric like "IQ" that we can apply to AI systems, and even if we did have the metric, at this point we wouldn't really know it's range, whether to expect it to be a linear function or not, etc.
That said... does it seem clear that we are making some kind of progress? I'd say yes, but with the admission that that's somewhat subjective. In particular, even though we know that current AI systems are getting better at doing whatever they do, we don't really know if that meaningfully counts as progress towards AGI. It is possible, for example, that LLM approaches will hit a ceiling in their development, and that the much feared "GPT 17" really won't even be a thing. And it may likewise turn out that the progress made on LLM's doesn't turn out to contribute much at all to whatever the eventual mechanism behind AGI turns out to be.
Is that the way I expect things to play out? Not sure. I have a hunch that LLM's will prove to be insufficient to qualify as AGI on their own, but that they may well serve as an element of a modular / hybrid system that includes LLMs, reasoners using deduction, induction, and abduction, evolutionary algorithms, and FSM-knows what else.
I don't know if we're close, but it seems like a lot of people have an implicit assumption that AGI means we created an artificial person. When in reality whatever intelligent things we create in the digital world are not going to behave like people. I mean, an AGI didn't poop its pants in 3rd grade or have a disapproving father or get ridiculed by a bully or a million other things that people normally go through in their lives that lead to their quirky personalities. It's going to be an anti-septically sheltered weirdo with super-human reasoning and a whole lot of unfunny jokes.
Related (and since you qualify the answers as "definitely yes" or "definitely no" I don't think it subtracts from what you say, but rather just adds to it:
> When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
I fully agree. Predicting future events is even more difficult than it seems and we constantly fail.
The only thing we can reasonably say about the state of AI is that we are closer to so-called AGI. We're seeing exciting progress and that it turn is attracting more efforts in the field, potentially leading to an acceleration.
But we could hit a wall, for many reasons, the field is almost entirely filled by unknown unknowns.
Nuclear fusion, Quantum Computing, Nanotech, Genetic Engineering, all of those fields are still evolving, slowly, because of technological and scientifical roadblocks.
I think the bigger issue I have is that I think we risk treating them as if they were AGI and that almost then makes them so but with greater risk. It's also a bit of a shifting goal posts in some ways too.
We're not close and it probably doesn't really matter. I mean, yes I want a real world Jarvis but that's not really what's important.
What's important is that AI will quickly replace toil in white collar work. Unfortunately, a lot of people make entire careers of that kind of work. "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing.
We're going to have to reckon with the consequences of AI long before AGI becomes real.
This argument is so common and I don’t get it. It lacks any context to historical advancements of technology.
This will only be good for our economy, any argument against is arguing against the entire trajectory of human technological advancements and its benefits to economic growth. As such, this argument requires a lot more evidence than the plainly obvious: that jobs will be lost.
Seriously, who wants to work a job that a machine can do better, cheaper, and more efficiently? Work for the sake of work is not meaningful.
This is creative destruction and it is the engine that drives economic growth. It can only be a good thing.
I agree that "AGI" and "labor transformation" are two different issues.
I do want to pick on what I think is a bit of hype around the potential to eliminate whole classes of work, though.
First, in the weeds: I keep seeing this reference to script writing, which I assume is a reference to the WGA strike. But as https://www.gq.com/story/writers-strike-2023-wga-explained explains, the big technological transformation driving this dispute isn't the risk of generative AI; it's the changes in royalty payments created by streaming platforms.
I'm pretty skeptical of claims that generative models can replace script writers. Like, maybe for some really, really crappy shows? But I think "My ChatGPT could write that" is the 21st century version of looking at a Rauschenberg and saying, "My 5 year old could paint that." Good luck, I guess?
More generally: Labor productivity growth has remained fairly constant over the last 40 years or so. So anyone who thinks generative AI is going to totally upend productivity and the job market is implicitly suggesting it's a bigger technological change than the Internet--which didn't really do that!
I won't soon forget how as a tools programmer at a game studio, I noticed a couple of the writers spent a big portion of their day churning text from the design team's word docs through excel macros to generate CSV files consumed by our build system. I looked at that and went "oh, I can just automate this", and replaced all that busywork with a tiny C# program that would just do it correctly all the time automatically so nobody had to do it by hand anymore.
Only occurred to me much later that now they didn't have much work to do and were at a higher risk of being laid off! A good lesson to think about all the implications of the software I'm writing. We've seen AI-adjacent tooling getting leveraged to eliminate more of that category of work in games already, like automated generation of NPC "barks".
This type of thinking really needs to look beyond what the hype train is on with OpenAI right now and look at some of the more groundbreaking work going on.
Deepmind has quite a bit of work outside the language area that has a lot of potential. Peter Velockovic, Matt Botvinick, Chelsea Finn, Taco Cohen, Micheal Bronstein, et al are all working completely outside the arena of NLP, but they have much more groundbreaking work than what everyone seems laser focused on right now.
There are tons of examples really, but for instance: their agents that have emergent properties of teamwork, as well as their work on developing entirely new mathematics in some of the hardest fields (knot theory and representation theory).
Creating new mathematics and proving theorems is probably the most creative form of human expression there is, so the fact that AI can achieve this is quite a big leap. Combining this with the deficits in LLMs can construct something that has much fewer flaws.
While some of the hype around language models is fading now, we will probably see another resurgence of interest by the media when Deepmind releases a system with these capabilities underneath a LM.
These LMs have been around for many years, and are just now getting mass attention. So it will likely take some more time of these much more sophisticated systems being solely in the academic arena before people become aware of what is really going on in AI (company products typically lag very far behind what academia is capable of).
However, soon there will be products that have attached a linguistic interface to these types of systems, and will likely yield some really impressive capabilities.
> "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing.
Something tells me that the low end creatives have a lot more to worry about than the paper pushers. If the past fifty years are any indication, there is a lot more inertia when it comes to replacing existing jobs than there is replacing something you would contract out.
Are you thinking there is a layer of management that will be AI or majority AI? What’re you envisioning?
I think we may see the best at creative tasks further separating themselves from the pack, these AI systems are best used by experts who can filter and quality check the AI output. I can see software engineering/dev becoming one extremely skilled dev managing a fleet of LLMs weaving together their own work and AIs’ work.
>Unfortunately, a lot of people make entire careers of that kind of work. "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing.
Don't forget programming jobs. I wouldn't classify programmers as "low end" as that's sort of an offensive categorization.
But among the set of careers you have chosen as a target for replacement... Programming is among that group. I mean LLMs are obviously translating English to code in a way that was never seen before.
Given that you're a programmer on a site that is mostly made up of programmers I wonder why that specific career was not mentioned. Seems a little convenient to me.
I'm not saying this is you, but I feel there is a lot of hostile defensiveness on the capabilities of AI because the AI potentially trivializes our abilities as programmers. If we are among the group of careers you labeled as "low level" then are we not "low level" ourselves? Do we have the fortitude and objectiveness to accept that?
The author's definition of "strong AI" is incredibly weak.
> Theory of Mind: These AI systems are capable of understanding mental states and can simulate the behavior of other agents.
GPT models can do this.
> Self-Aware
Why is that a requirement?
> These AI systems are capable of learning any intellectual task that a human can.
Again, GPT models appear to do this (at least for text-based tasks).
That being said, I think GPT models do have some limitations that prevent them for learning any intellectual task a human can do. I wrote a blog post a few days ago that explored trying to "teach" ChatGPT to perform binary addition [0]. My conclusion was that, when ChatGPT appears to solve a logical problem, that solution is just a regurgitation of a low-entropy example generalized from its training data. But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems.
I don't understand how you can write a whole piece on Transformers failing to do arithmetics without mentioning the tokenizer used in GPT, which makes it more or less impossible for the model.
The reason is imo probably more along the lines of that the model can't see the actual numbers you're giving it.
When you provide GPT3 with the number "10011111110100011100011011110010", it actually sees [3064, 26259, 1157, 8784, 18005, 1157, 18005, 8784, 1157, 3064, 940] [see 0].
The number isn't even split up into even parts. Even if it knew the Peano axioms, it wouldn't have a chance because it can't 'see' the actual numbers that you're feeding it. Hence it can only estimate these operations based on combination pairs it has seen in training data, and it seems pretty obvious why this can't be a robust basis for doing arithmetic.
It might be more trainable if the the bits where space delimited so that only the tokens [ 1] and [ 0] were in use.
As it is, it doesn't really understand the contents of the tokens and manipulating [405, 1157, 8784, 486, 8298, 486, 18005, 2388, 486, 486, 1157, 8784, 16] is not easily trained - especially if there is a different bit sequence. Note also the misalignments of the tokens for bit length that can make it more difficult to work with.
Arithmetic is a known-bad example because of how the tokenization works. Similar to calling a human unintelligent because we can't recognize bat calls.
Additionally 'can't do x, and is therefore a lookup table' is a very poor logical leap. Are humans who can't add just lookup tables as well?
Some sort of self-awareness is required for an AI system interacting with the real world physically - i.e. via controlling a robot or a vehicle - safely and to react appropriately for situations it has not been taught for.
> But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems.
IIRC, the problem with current GPT AI models is tokenization. Basically, words are long and somewhat visually distinct with clear separation, whereas numbers are far harder to "convert" into tokens the model can use.
In the end I believe that this problem will eventually be solved by applying enough compute power, as will the problem with "self awareness" - run a GPT model in a continuous loop that allows self training.
The real and actually very hard problem IMO is adversarial training and inputs. In humans, the limits of a human brain in the speed of interaction and the number of other actors one brain can interact with act as a natural hard performance ceiling as well as a quantity barrier for too much adversarial input (you can only get fed bullshit 24 person-hours a day), and yet we're seeing adversarial inputs (aka propaganda) be very effective to the tune that 40% of Americans believe that the 2020 election was stolen or manipulated [1].
An AI system that can interact with hundreds of thousands of people simultaneously? Microsoft proved how fast that can go sideways with Tay 2016, when 4chan and other trolls managed to turn the bot into a raging Nazi in not even a day worth of effort [2]. Now imagine you have an AI running airspace travel control and someone convinces it that a plane got hijacked by terrorists or to lead planes onto a collision course...
the linked blog is yet another instance of making false inferences from the false assumption of the model working at a letter/symbol level. it cannot see letters. it cannot see the 1 or 0 in 11010 either. (note that a request to simply reverse a word would illustrate this usually, but some glue code has been added to do it)
It is telling though, that a year ago myself and I think most programmers would have just quickly said "nope definitely not", whereas now you kind of have to get long winded to say no, and many of us are not saying no but maybe
AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it.
What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such?
How close are we to AGI and do people actually understand what that means? Too often people use the term as if General were 'ubiquitious' as in a broad application of AI instead of one that is inditinguishable from human intelligence.
We definitely don't have consensus about either "general" or "intelligence" (and if mind-uploads ever enter the discussion, then also about "artificial").
For me, "general" is a sliding scale, not a yes-or-no boolean, so I think it's fine to say that GPT3, let alone later versions, is "general" in the domain of text — it has more general knowledge than I have, that's for sure!
Likewise "intelligent", though for me that's a vector value: [i_0, i_1, …, i_m] where each i is the ability at some aspect of intelligence.
In fact, I'd go further and say that you can define "generality" with respect to such a vector: if the intelligence vector is a normalised score 0-1 for each value, then generality could be easily defined by the statistics of that array.
As even GPT-4 seems to be better than the layperson and worse than an expert at everything (textual) it has been tested on, I'm not sure if it's more or less general than a human by this standard.
And you may wish to define intelligence by ability to learn from small data sets rather than large, at which point we then need to consider the difference between the initial training, the RLHF, and any fine-tuning; and how this compares to evolution, education, and work experience.
We struggle to define the traits, characteristics, properties of intelligence, sentience or awareness. Nor can we locate the physical processes, exactly, by which these things come into being in a human brain. Some voodoo to do with information processing in an embodied system as an agent trying to survive in the world, maybe, if you're a materialist.
We know approximately nothing about the configuration that a state of matter would need, to attain intelligent, sentient, aware properties like we have. Other than that something extremely like a human brain (including non-human animal brains) seem to possess these properties in some way. Certain configurations in machines are starting to exhibit the first trait, at least, intelligence. Arguably.
I don't think we can even rule out that AGI might, kind of just accidentally happen, in some sufficiently complex self-feedback machine learning-based information-processing task. Since that's one of the main hypotheses for how we came about to think.
Just a gut feeling tells me nothing we've built is anywhere near large enough for that. But the supposition that intelligence, sentience, awareness, are tied to big data, big processing, and biological parallelism -- is just that, a supposition. What if the secret of it all boils down to something that can be described mathematically on one page (my other gut feeling), which happens to be physically realized in some way in our heads? Can't be ruled out, either.
I can't really see how to set any bounds on anything with these questions. How many bits does it take to hold a mind?
Why do we need to be? Say human intelligence evolved so that we model real world events such a mammoth hunt mentally ahead of time and use simulated outcomes to craft weapons / preposition different hunters based on their skills. Would AI necessarily need to hunt or worry about energy intake/self preservation/producing offspring? It can be powered by direct photosynthesis (solar panels), hibernate when energy is in short supply and make backups to safeguard against future physical damage. Seems like we should think through the tasks that we need AI to help us with - climate change, medicine, space exploration, etc - and design systems for such tasks as efficiently as possible rather than trying to monkey ourselves. And sure, some day AI may develop its own purposes that are different from its origin, just like humans decided they like music. Doesn't mean that a caveman shouldn't have dedicated his attention to the next meal rather than worrying about being sophisticated.
The simple answer is “go figure out how solve climate change” is a much easier direction to give an AI than figuring out what to do then building a bunch of little specific ones.
Really all an AGI is an AI smart enough to build and use other smaller AIs, or at least that’s a very common guess as to the architecture that’ll end up working.
> This captures the crux of the problem in that we claimed we’d have True AI when a computer could beat a human in chess. But when that occurred with Deep Blue, and we saw the code, it was explainable and therefore not true AI. Then we moved the goalpost and said that if a computer could beat a human in Go, then we’d have True AI. When that occurred with AlphaGo and we saw the code, we moved the goalposts yet again.
We believed that chess and go proficiency were good proxies for true AI. They weren't. They were proxies for advancement, but the underlying issue is that we don't know how much we don't know about the human mind.
Every AI advancement helps us learn more about our minds, but the amount we don't know is still unknown.
The moving of the goalposts was not done because "we saw the code" it was because Deep Blue and Alpha Go were obviously not AI. Again, we were just wrong about those tasks being good proxies.
I'm not sure we were wrong about Go. The general approach that conquered Go just a few years later led to something passing the Turing test. Give it a few more years, see what happens.
No those were never official goal posts. The one goal post was official and famous was the Turing test that measured if a human could tell the difference between a text chat generated by an AI or one generated by a human. https://en.m.wikipedia.org/wiki/Turing_test
Literally the inventor of the Turing machine came up with this test so this was always the north star goal post since computing began. It's a way higher bar then any of the nameless goal posts you mentioned.
Unfortunately this goal post was also just moved recently with LLMs.
Even if say we are close to “AGI” it’s going to be something significantly different from human and animal intelligence. One of the most obvious differences that I don’t see people bring up:
1. ChatGPT can only respond to an input. If you left it alone it would literally do nothing. It cannot, generate, create thoughts and choose what stimuli to respond to.
This posits that the human brain is like a dynamical system. You switch it on and it keeps going on forever, there is also no hardline between learning and inference like DL. ChatGPT and others etc, feel very much like a digital system. It can only respond to inputs provided, there is a clear demarcation between the learning stage and the inference stage
Note: it is still possible that these AI systems will reach human like performance in a variety of tasks. But they will seem very weird and different from the intelligent systems we are exposed to in Nature
My current model of AI is that we're applying massive amounts of compute to force a function approximator to learn a function. To solve this function, there is likely a huge mismatch between the learned results and the internal logic used to generate those results. I believe that it is possible that the mismatch is several orders of magnitude, and thus it might be that the cognitive core could be quite a superintelligence hobbled by the layers around it.
It is quite possible that there are cognitive strategies at use that haven't been invented yet in the outside world. At this point, it would require better training data, or even more compute, to let that intelligence reach the outside world unfiltered.
Only time will tell if any of this is true, it's just speculation on my part at present.
> AI is the development of machines with the ability to think and act like humans
What we have today is machine learning... not this. LLM's are neat, transformers are a really useful innovation, etc. Some people like to leap to conclusions and claim that this is intelligence beyond human comprehension and it will learn to destroy is all. I think that's a bit much.
If the goal of AGI is to be this we're nowhere near it. And I suspect there are plenty of people in this "AI" space that would not see it as a goal of the research. An expert system that can reason in a particular domain would be a huge advance and a useful tool.
There's no need to also create autonomous agents that think and act like humans. We're pretty terrible at that on the whole.
It seems like an AGI would just be a meta-machine-learning model, meaning it's a model whose whole goal is to think up new ways of training models. Eventually, it would figure out a way to either incorporate the smaller, specialized subtasks into itself and piecemeal upgrade itself. Oh, and it needs permission to use $MEGA_DOLLARS to run the computations necessary for training. So, it would probably need a side hussle. Like search ads or something.
Right now, humans are either the bailing wire or the chewing gum in this situation, but they hold the purse strings and GPU farms. Humans are the ones thinking up new training strategies, curating the learning datasets, and building the cluster and cloud and datacenter systems that can handle that much data.
Past it. AGI was publicly unveiled on November 30, 2022 and currently has two hundred million users. It is capable of general abstract thinking on any subject regardless of whether it's already in its training set or not. It meets my definitions of AGI.
If you observe toddlers, at the early stages of their development you can see the "inner voice" at play, their lips moving as they reason through scenarios whilst playing with toys (and deaf/mute children similarly signing with their hands under the table).
The same work is being done now with LLMs, where you can prompt-engineer models to reason about its generated text.
Virtual personal assistant isn't exactly a narrow definition. I'd expect that's a big role to play and we'll make AGI systems do that too.
The strong AI definitions are also pretty biased towards human experiences. I wouldn't assume future innovations are necessarily human-like. It's hard to imagine a completely new thing until it gets here though.
"Learning any task" is also, uh, quite expansive - does the AI lose its AGI badge if the internet finds something it sucks at? (spoiler: it will suck at something)
Its possible that the AGI obsession will fizzle. The analogy is fruitful to some extent as an inspiration. But it is too remote and ambitious to work as a roadmap.
There is precedence. People called certain algorithms neural networks, but they are very far from real neurons and they are not getting closer. Why should they?
People will keep developing "AI" algorithms but it will be along the natural pathways that underlies these mathematical structures (which are rather simplistic) and focusing on problems that people find important.
Wrt "No True Autonomy Fallacy" and the examples provided underneath, I am reminded of this Tuned Deck magic trick example used by Daniel Dennett to explain consciousness.
Mass media has so heavily conveyed the fact that humans are transcendent of the laws of physics, the weeping tear that brings back the dying or the true love bond that defeats the machines. This trope is so tired, but man do our narcissistic meat brains love it.
AGI or not, I think people conditioned their whole lives that humans are transcendent are starting to get smashed through the wringer of reality, making it hard to discern honest takes from desperate coping takes.
What I would’ve preferred in this article which is largely long winded definitions and summaries of computing history that are much better explained on Wikipedia would be a workable, testable definition of AGI. How will we know when we’ve gotten there? Frankly, I don’t care about the Turing test. There’s several versions of it, and it’s not a general test of intelligence anyways. It’s a parlor game.
I don't think a single 'what is AGI' answer will satisfy everyone, someone will always have a complaint about something.
I personally recommend we make a bunch of different test or 'classifications' that can be measured. Then we test each AI system against these classifications. This isn't to define if its AGI or not, but it's capabilities. For example, if a human took some of these tests they could also fail because the human body/mind is not capable.
Just as an example, if we said "Flying is what birds do", then every plane would fail even though the capabilities of planes are far more useful to humans.
I see this as a much better framework than the nebulous goal of 'AGI' itself.
[+] [-] mindcrime|2 years ago|reply
So as of today, we don't really have any kind of simple metric like "IQ" that we can apply to AI systems, and even if we did have the metric, at this point we wouldn't really know it's range, whether to expect it to be a linear function or not, etc.
That said... does it seem clear that we are making some kind of progress? I'd say yes, but with the admission that that's somewhat subjective. In particular, even though we know that current AI systems are getting better at doing whatever they do, we don't really know if that meaningfully counts as progress towards AGI. It is possible, for example, that LLM approaches will hit a ceiling in their development, and that the much feared "GPT 17" really won't even be a thing. And it may likewise turn out that the progress made on LLM's doesn't turn out to contribute much at all to whatever the eventual mechanism behind AGI turns out to be.
Is that the way I expect things to play out? Not sure. I have a hunch that LLM's will prove to be insufficient to qualify as AGI on their own, but that they may well serve as an element of a modular / hybrid system that includes LLMs, reasoners using deduction, induction, and abduction, evolutionary algorithms, and FSM-knows what else.
[+] [-] titzer|2 years ago|reply
[+] [-] reitanqild|2 years ago|reply
> When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
(Clarkes first law, copied from https://en.wikipedia.org/wiki/Clarke%27s_three_laws)
[+] [-] stephc_int13|2 years ago|reply
The only thing we can reasonably say about the state of AI is that we are closer to so-called AGI. We're seeing exciting progress and that it turn is attracting more efforts in the field, potentially leading to an acceleration.
But we could hit a wall, for many reasons, the field is almost entirely filled by unknown unknowns.
Nuclear fusion, Quantum Computing, Nanotech, Genetic Engineering, all of those fields are still evolving, slowly, because of technological and scientifical roadblocks.
[+] [-] mpwouden|2 years ago|reply
[+] [-] davewritescode|2 years ago|reply
What's important is that AI will quickly replace toil in white collar work. Unfortunately, a lot of people make entire careers of that kind of work. "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing.
We're going to have to reckon with the consequences of AI long before AGI becomes real.
[+] [-] qudat|2 years ago|reply
This will only be good for our economy, any argument against is arguing against the entire trajectory of human technological advancements and its benefits to economic growth. As such, this argument requires a lot more evidence than the plainly obvious: that jobs will be lost.
Seriously, who wants to work a job that a machine can do better, cheaper, and more efficiently? Work for the sake of work is not meaningful.
This is creative destruction and it is the engine that drives economic growth. It can only be a good thing.
[+] [-] md_|2 years ago|reply
I do want to pick on what I think is a bit of hype around the potential to eliminate whole classes of work, though.
First, in the weeds: I keep seeing this reference to script writing, which I assume is a reference to the WGA strike. But as https://www.gq.com/story/writers-strike-2023-wga-explained explains, the big technological transformation driving this dispute isn't the risk of generative AI; it's the changes in royalty payments created by streaming platforms.
I'm pretty skeptical of claims that generative models can replace script writers. Like, maybe for some really, really crappy shows? But I think "My ChatGPT could write that" is the 21st century version of looking at a Rauschenberg and saying, "My 5 year old could paint that." Good luck, I guess?
More generally: Labor productivity growth has remained fairly constant over the last 40 years or so. So anyone who thinks generative AI is going to totally upend productivity and the job market is implicitly suggesting it's a bigger technological change than the Internet--which didn't really do that!
So...dunno. That's a bold claim!
[+] [-] kevingadd|2 years ago|reply
Only occurred to me much later that now they didn't have much work to do and were at a higher risk of being laid off! A good lesson to think about all the implications of the software I'm writing. We've seen AI-adjacent tooling getting leveraged to eliminate more of that category of work in games already, like automated generation of NPC "barks".
[+] [-] chaxor|2 years ago|reply
Deepmind has quite a bit of work outside the language area that has a lot of potential. Peter Velockovic, Matt Botvinick, Chelsea Finn, Taco Cohen, Micheal Bronstein, et al are all working completely outside the arena of NLP, but they have much more groundbreaking work than what everyone seems laser focused on right now.
There are tons of examples really, but for instance: their agents that have emergent properties of teamwork, as well as their work on developing entirely new mathematics in some of the hardest fields (knot theory and representation theory).
Creating new mathematics and proving theorems is probably the most creative form of human expression there is, so the fact that AI can achieve this is quite a big leap. Combining this with the deficits in LLMs can construct something that has much fewer flaws.
While some of the hype around language models is fading now, we will probably see another resurgence of interest by the media when Deepmind releases a system with these capabilities underneath a LM. These LMs have been around for many years, and are just now getting mass attention. So it will likely take some more time of these much more sophisticated systems being solely in the academic arena before people become aware of what is really going on in AI (company products typically lag very far behind what academia is capable of).
However, soon there will be products that have attached a linguistic interface to these types of systems, and will likely yield some really impressive capabilities.
[+] [-] II2II|2 years ago|reply
Something tells me that the low end creatives have a lot more to worry about than the paper pushers. If the past fifty years are any indication, there is a lot more inertia when it comes to replacing existing jobs than there is replacing something you would contract out.
[+] [-] dpflan|2 years ago|reply
I think we may see the best at creative tasks further separating themselves from the pack, these AI systems are best used by experts who can filter and quality check the AI output. I can see software engineering/dev becoming one extremely skilled dev managing a fleet of LLMs weaving together their own work and AIs’ work.
[+] [-] byyyy|2 years ago|reply
Don't forget programming jobs. I wouldn't classify programmers as "low end" as that's sort of an offensive categorization.
But among the set of careers you have chosen as a target for replacement... Programming is among that group. I mean LLMs are obviously translating English to code in a way that was never seen before.
Given that you're a programmer on a site that is mostly made up of programmers I wonder why that specific career was not mentioned. Seems a little convenient to me.
I'm not saying this is you, but I feel there is a lot of hostile defensiveness on the capabilities of AI because the AI potentially trivializes our abilities as programmers. If we are among the group of careers you labeled as "low level" then are we not "low level" ourselves? Do we have the fortitude and objectiveness to accept that?
Probably not.
[+] [-] aanya_dawkins|2 years ago|reply
[deleted]
[+] [-] rcme|2 years ago|reply
> Theory of Mind: These AI systems are capable of understanding mental states and can simulate the behavior of other agents.
GPT models can do this.
> Self-Aware
Why is that a requirement?
> These AI systems are capable of learning any intellectual task that a human can.
Again, GPT models appear to do this (at least for text-based tasks).
That being said, I think GPT models do have some limitations that prevent them for learning any intellectual task a human can do. I wrote a blog post a few days ago that explored trying to "teach" ChatGPT to perform binary addition [0]. My conclusion was that, when ChatGPT appears to solve a logical problem, that solution is just a regurgitation of a low-entropy example generalized from its training data. But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems.
0: https://nobuteru.substack.com/p/can-chatgpt-learn-to-add
[+] [-] sva_|2 years ago|reply
The reason is imo probably more along the lines of that the model can't see the actual numbers you're giving it.
When you provide GPT3 with the number "10011111110100011100011011110010", it actually sees [3064, 26259, 1157, 8784, 18005, 1157, 18005, 8784, 1157, 3064, 940] [see 0].
The number isn't even split up into even parts. Even if it knew the Peano axioms, it wouldn't have a chance because it can't 'see' the actual numbers that you're feeding it. Hence it can only estimate these operations based on combination pairs it has seen in training data, and it seems pretty obvious why this can't be a robust basis for doing arithmetic.
0. https://platform.openai.com/tokenizer
[+] [-] shagie|2 years ago|reply
https://platform.openai.com/tokenizer
is tokenized as: It might be more trainable if the the bits where space delimited so that only the tokens [ 1] and [ 0] were in use.As it is, it doesn't really understand the contents of the tokens and manipulating [405, 1157, 8784, 486, 8298, 486, 18005, 2388, 486, 486, 1157, 8784, 16] is not easily trained - especially if there is a different bit sequence. Note also the misalignments of the tokens for bit length that can make it more difficult to work with.
[+] [-] sdenton4|2 years ago|reply
Additionally 'can't do x, and is therefore a lookup table' is a very poor logical leap. Are humans who can't add just lookup tables as well?
[+] [-] mschuster91|2 years ago|reply
Some sort of self-awareness is required for an AI system interacting with the real world physically - i.e. via controlling a robot or a vehicle - safely and to react appropriately for situations it has not been taught for.
> But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems.
IIRC, the problem with current GPT AI models is tokenization. Basically, words are long and somewhat visually distinct with clear separation, whereas numbers are far harder to "convert" into tokens the model can use.
In the end I believe that this problem will eventually be solved by applying enough compute power, as will the problem with "self awareness" - run a GPT model in a continuous loop that allows self training.
The real and actually very hard problem IMO is adversarial training and inputs. In humans, the limits of a human brain in the speed of interaction and the number of other actors one brain can interact with act as a natural hard performance ceiling as well as a quantity barrier for too much adversarial input (you can only get fed bullshit 24 person-hours a day), and yet we're seeing adversarial inputs (aka propaganda) be very effective to the tune that 40% of Americans believe that the 2020 election was stolen or manipulated [1].
An AI system that can interact with hundreds of thousands of people simultaneously? Microsoft proved how fast that can go sideways with Tay 2016, when 4chan and other trolls managed to turn the bot into a raging Nazi in not even a day worth of effort [2]. Now imagine you have an AI running airspace travel control and someone convinces it that a plane got hijacked by terrorists or to lead planes onto a collision course...
[1] https://www.newsweek.com/40-americans-think-2020-election-st...
[2] https://en.wikipedia.org/wiki/Tay_(chatbot)
[+] [-] carrolldunham|2 years ago|reply
[+] [-] the_gipsy|2 years ago|reply
> Why is that a requirement?
And what does self-aware even mean? A lot of animals pass the mirror test.
[+] [-] moffkalast|2 years ago|reply
> Why is that a requirement?
Not to mention that this cannot be proven at all, even for humans.
[+] [-] md_|2 years ago|reply
[+] [-] jackmott42|2 years ago|reply
AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it.
What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such?
Gonna be fun.
[+] [-] mpwouden|2 years ago|reply
[+] [-] ben_w|2 years ago|reply
For me, "general" is a sliding scale, not a yes-or-no boolean, so I think it's fine to say that GPT3, let alone later versions, is "general" in the domain of text — it has more general knowledge than I have, that's for sure!
Likewise "intelligent", though for me that's a vector value: [i_0, i_1, …, i_m] where each i is the ability at some aspect of intelligence.
In fact, I'd go further and say that you can define "generality" with respect to such a vector: if the intelligence vector is a normalised score 0-1 for each value, then generality could be easily defined by the statistics of that array.
As even GPT-4 seems to be better than the layperson and worse than an expert at everything (textual) it has been tested on, I'm not sure if it's more or less general than a human by this standard.
And you may wish to define intelligence by ability to learn from small data sets rather than large, at which point we then need to consider the difference between the initial training, the RLHF, and any fine-tuning; and how this compares to evolution, education, and work experience.
[+] [-] knallfrosch|2 years ago|reply
[+] [-] ftxbro|2 years ago|reply
no because it means something different to each person (both its definition and its implications)
[+] [-] retrac|2 years ago|reply
We know approximately nothing about the configuration that a state of matter would need, to attain intelligent, sentient, aware properties like we have. Other than that something extremely like a human brain (including non-human animal brains) seem to possess these properties in some way. Certain configurations in machines are starting to exhibit the first trait, at least, intelligence. Arguably.
I don't think we can even rule out that AGI might, kind of just accidentally happen, in some sufficiently complex self-feedback machine learning-based information-processing task. Since that's one of the main hypotheses for how we came about to think.
Just a gut feeling tells me nothing we've built is anywhere near large enough for that. But the supposition that intelligence, sentience, awareness, are tied to big data, big processing, and biological parallelism -- is just that, a supposition. What if the secret of it all boils down to something that can be described mathematically on one page (my other gut feeling), which happens to be physically realized in some way in our heads? Can't be ruled out, either.
I can't really see how to set any bounds on anything with these questions. How many bits does it take to hold a mind?
[+] [-] aanya_dawkins|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] cat_plus_plus|2 years ago|reply
[+] [-] bbor|2 years ago|reply
Really all an AGI is an AI smart enough to build and use other smaller AIs, or at least that’s a very common guess as to the architecture that’ll end up working.
I give it till Christmas
[+] [-] ballenf|2 years ago|reply
We believed that chess and go proficiency were good proxies for true AI. They weren't. They were proxies for advancement, but the underlying issue is that we don't know how much we don't know about the human mind.
Every AI advancement helps us learn more about our minds, but the amount we don't know is still unknown.
The moving of the goalposts was not done because "we saw the code" it was because Deep Blue and Alpha Go were obviously not AI. Again, we were just wrong about those tasks being good proxies.
[+] [-] jackmott42|2 years ago|reply
[+] [-] byyyy|2 years ago|reply
Literally the inventor of the Turing machine came up with this test so this was always the north star goal post since computing began. It's a way higher bar then any of the nameless goal posts you mentioned.
Unfortunately this goal post was also just moved recently with LLMs.
[+] [-] sashank_1509|2 years ago|reply
1. ChatGPT can only respond to an input. If you left it alone it would literally do nothing. It cannot, generate, create thoughts and choose what stimuli to respond to.
This posits that the human brain is like a dynamical system. You switch it on and it keeps going on forever, there is also no hardline between learning and inference like DL. ChatGPT and others etc, feel very much like a digital system. It can only respond to inputs provided, there is a clear demarcation between the learning stage and the inference stage
Note: it is still possible that these AI systems will reach human like performance in a variety of tasks. But they will seem very weird and different from the intelligent systems we are exposed to in Nature
[+] [-] mikewarot|2 years ago|reply
It is quite possible that there are cognitive strategies at use that haven't been invented yet in the outside world. At this point, it would require better training data, or even more compute, to let that intelligence reach the outside world unfiltered.
Only time will tell if any of this is true, it's just speculation on my part at present.
[+] [-] agentultra|2 years ago|reply
What we have today is machine learning... not this. LLM's are neat, transformers are a really useful innovation, etc. Some people like to leap to conclusions and claim that this is intelligence beyond human comprehension and it will learn to destroy is all. I think that's a bit much.
If the goal of AGI is to be this we're nowhere near it. And I suspect there are plenty of people in this "AI" space that would not see it as a goal of the research. An expert system that can reason in a particular domain would be a huge advance and a useful tool.
There's no need to also create autonomous agents that think and act like humans. We're pretty terrible at that on the whole.
[+] [-] titzer|2 years ago|reply
Right now, humans are either the bailing wire or the chewing gum in this situation, but they hold the purse strings and GPU farms. Humans are the ones thinking up new training strategies, curating the learning datasets, and building the cluster and cloud and datacenter systems that can handle that much data.
[+] [-] logicallee|2 years ago|reply
[+] [-] opyate|2 years ago|reply
The same work is being done now with LLMs, where you can prompt-engineer models to reason about its generated text.
See https://youtu.be/wVzuvf9D9BU
This video captures some of the state-of-the-art currently. (E.g. see the video description links)
So, decoders are getting their own little inner voices. A small little step towards AGI, for sure.
[+] [-] vsareto|2 years ago|reply
The strong AI definitions are also pretty biased towards human experiences. I wouldn't assume future innovations are necessarily human-like. It's hard to imagine a completely new thing until it gets here though.
"Learning any task" is also, uh, quite expansive - does the AI lose its AGI badge if the internet finds something it sucks at? (spoiler: it will suck at something)
[+] [-] nologic01|2 years ago|reply
There is precedence. People called certain algorithms neural networks, but they are very far from real neurons and they are not getting closer. Why should they?
People will keep developing "AI" algorithms but it will be along the natural pathways that underlies these mathematical structures (which are rather simplistic) and focusing on problems that people find important.
[+] [-] passion__desire|2 years ago|reply
https://jessegalef.com/2011/01/09/the-tuned-deck/
https://www.youtube.com/watch?v=LVg23QRJD70
[+] [-] Workaccount2|2 years ago|reply
AGI or not, I think people conditioned their whole lives that humans are transcendent are starting to get smashed through the wringer of reality, making it hard to discern honest takes from desperate coping takes.
[+] [-] janalsncm|2 years ago|reply
[+] [-] pixl97|2 years ago|reply
I personally recommend we make a bunch of different test or 'classifications' that can be measured. Then we test each AI system against these classifications. This isn't to define if its AGI or not, but it's capabilities. For example, if a human took some of these tests they could also fail because the human body/mind is not capable.
Just as an example, if we said "Flying is what birds do", then every plane would fail even though the capabilities of planes are far more useful to humans.
I see this as a much better framework than the nebulous goal of 'AGI' itself.