Author here: I'm grateful for the comments; thanks especially for interesting references.
Context for the article: I'm working on an ambitious long-term project to write a book about consciousness from a scientific and analytic (versus, say, a meditation-oriented) perspective. I didn't write this fact in the article, but what I'd love to happen is that I meet people with a similar optimistic perspective, and to learn and improve my communication skills via follow-up conversations.
If anyone is interested in chatting more about the topic of the article, please do email me. My email is in my HN profile. Thanks!
Dennett has walked that path before. In “Consciousness Explained,” a 1991 best-seller, he described consciousness as something like the product of multiple, layered computer programs running on the hardware of the brain. [...]
Regarding the multiple layers: the most interesting thoughts I read about theories of a mind, are the books by Marvin Minsky, namely: The Society of Mind and The Emotion Machine which should be more widely known.
"The Experience Machine: How Our Minds Predict and Shape Reality" could be interesting to you. The framework it presents tries to explain attention and world models, and does so quite convincingly in my opinion.
Also a must read: How the Mind Works, Steven Pinker.
Pinker advocates for a model of mind that has multiple streams of consciousness. In Pinker's model of mind, there are multiple agents constructing models of the world in parallel, each trying to predict future states from current state plus current input data. A supervisory process then selects the model that has made the best prediction of current state in the recent past for use when reacting in the current moment. The supervisor process is free to switch between models on the fly as more data comes in.
Pinker grounds his model of mind in curious observations of what people remember, and how our short term memories change over a period of sometimes many seconds as our mind switches between different agent interpretations of what's going on. Witnesses are notoriously unreliable. Pinker concerns himself with why and how witnesses are unreliable, not for legal reasons, but for how those unreliabilities might reveal structure of mind. Pinker's most interesting observation (I think) is that what we seem to remember is output from the models rather than the raw input data; and that what we remember seeing can change dramatically over a period of many seconds as the supervisory process switches between models. Notably, we seem to remember, in short term memory, details of successful models that make "sense", even when those details contradict with what we actually saw. And when those details are moved to longer-term memory, the potentially inaccurate details of model are what are committed.
In particular, I think there's a nice overlap between my description of self-awareness and yours. You mention a model capable of seeing its own output, but I take it a step further and indicate model capable of generating its own simulated actions and results. Curious what you think, thanks!
Too many people have written books about consciousness.
There's much tail-chasing in that space, all the way back to Aristotle.
Write one about common sense. Current AI sucks at common sense.
We can't even achieve the level of common sense of a squirrel yet.
Working definition of common sense: getting through the next 30 seconds of life without a major screwup.
Something that strikes me about this model is that it's bottom up - sensory data feeds in in its entirety, the action centre processes everything, makes a decision, sends a command to the motor centre.
There's a theory that real brains subvert this, and what we perceive is actually our internal model of our self/environment. The only data that makes it through from our sense organs is the difference between the two.
This kind of top-down processing is more efficient energy-wise but I wonder if it's deeper than that? You can view perception and action as two sides of the same coin - both are ways to modify your internal model to better fit the sensory signals you expect.
Anyway, I guess the point I'm making is you should be careful which way you point your arrows, and of designating a single aspect of a mind (the action centre) as fundamental. Reality might work very differently, and that maybe says something? I don't know haha.
I think you are correct. My understanding is that the brain uses sense data just to confirm predictions it has already made. The model in which the brain is just reacting to incoming sense data is outdated from what I understand.
I've lately begun to think of conciousness as the ability to read and react to one's own log output. I don't like hypothesis by analogy, but it seems an apt description for what conscious entities do. I just don't see anything mystical about it.
We evolved to be aware, model & react and to opportunistically control our environment. Loop 1.
Then we evolved to be aware, model, & & react and to opportunistically control our selves as bodies. Loop 2.
Then we evolved to be aware, model, & & react and to opportunistically control those activities too, our minds. In the process of being able to model & react to ourselves modelling & reacting to ourselves we completed a self-awareness of self-awareness loop. Loop 3.
I think this is a good explanation because what else is consciousness but knowing you are conscious? Self-aware that you are self-aware?
And it makes perfect sense as an evolutionary trajectory using greater levels of neural representation of our practical relationship with our environment, including ourselves, to create higher level survival options.
So this definition is both a functional and developmental explanation of the emergent phenomenon of consciousness.
Just as we only have partial access to information and control of our environment, our bodies, we are also limited to the degree we are aware and can control our mind. Introspection.
The next level up is “theory of mind”, “empathy”, etc. our ability to model & interact with others mind’s as individuals and groups, reciprocally. Loop 4. That created society and culture, living information that extends outside us and grows and lives beyond each of us.
Infusing our thoughts, and progressively thinking abilities into technology, that in theory and not too distant practice could have access to all parts of their own minds’ states, operations and design, would be Loop 5. When deeply conscious beings start, things will get interesting in the Sol system.
When people start talking about quantum waves or smart rocks or universes in the context of consciousness I feel a little ill. People like to “solve” unrelated mysteries by merging them and shouting “tada!” (“We are puzzled about how they built the pyramids… but we don’t know if there are aliens either, so obviously… these mysteries solve each other!”)
It's a really fascinating topic, but I wonder if this article could benefit from any of the extensive prior work in some way. There is actually quite a lot of work on AGI and cognitive architecture out there. For a more recent and popular take centered around LLMs, see David Shapiro.
Before that you can look into the AGI conference people like Ben Goertzel, Pei Wang. And actually the whole history of decades of AI research before it became about narrow AI.
I'd also like to suggest that creating something that truly closely simulates a living intelligent digital person is incredibly dangerous, stupid, and totally unnecessary. The reason I say that is because we already have superhuman capabilities in some ways, and the hardware, software and models are being improved rapidly. We are on track to have AI that is dozens if not hundreds of times faster than humans at thinking and much more capable.
If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
Don't get me wrong, I love AI and my whole life is planned around agents and AI. But I no longer believe it is wise to try to go all the way and create a "real" living digital species. And I know it's not necessary -- we can create effective AI agents without actually emulating life. We certainly don't need full autonomy, self preservation, real suffering, reproductive instincts, etc. But that is the goal he seems to be down in this article. I suggest leaving some of that out very deliberately.
> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down.
AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.
> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.
> (Pro-strong-AI)... This is basically a disbelief in the ability of physics to correctly describe what happens in the world — a well-established philosophical position. Are you giving up on physics?
This is a very strong argument. Certainly all the ingredients to replicate a mind must exist within our physical reality.
But does an algorithm running on a computer have access to all the physics required?
For example, there are known physical phenomena, such as quantum entanglement, that are not possible to emulate with classical physics. How do we know our brains are not exploiting these, and possibly even yet unknown, physical phenomena?
An algorithm running on a classical computer is executing in a very different environment than a brain that is directly part of physical reality.
> there are known physical phenomena, such as quantum entanglement
QC researcher here, strictly speaking, this is false. Clifford circuits can be efficiently simulated classically and they exhibit entanglement. The bottom line is we're not entirely sure where the (purported) quantum speedups come from. It might have something to do with entanglement, but it's not enough by itself.
Re: about mermin's device, im not sure why you think it can not be simulated classically when all of the dynamics involved can be explained by 4x4 complex matrices.
> Now the LLM can choose to switch, at its own discretion, back and forth between a talking and listening mode
How would it intelligently do this? What data would you train on? You don't have trillions words of text where humans wrote what they thought silently interwoven with what they wrote publicly.
History has shown over and over that hard coded ad hoc solutions to these "simple problems" never work to create intelligent agents, you need to train the model to do that from the start you can't patch in intelligence after the fact. Those additions can be useful, but they have never been intelligent.
Anyway, such a model I'd call "stream of mind model" rather than a language model, it would fundamentally solve many of the problems with current LLM where their thinking is reliant on the shape of the answer, while a stream of mind model would shape its thinking to fit the problem and then shape the formatting to fit the communication needs.
Such a model as this guy describes would be a massive step forward, so I agree with this, but it is way too expensive to train, not due to lack of compute but due to lack of data. And I don't see that data being done within the next decade if ever, humans don't really like writing down their hidden thoughts, and you'd need to pay them to generate data amounts equivalent to the internet...
Replying to: How would a model intelligently switch between listening or speaking modes? What data would you train on? (I'm the author of the parent article.)
It's a fair question, and I don't have all the answers. But for this question, there might be training data available from everyday human conversations. For example, we could use a speech-to-text model that's able to distinguish speakers, and look for points where one person decided to start speaking (that would be training data for when to switch modes). Ideally, the speech-to-text model would be able to include text even when both people spoke at once (this would provide more realistic and complete training data).
I've noticed that the audio mode in ChatGPT's app is good at noticing when I'm done speaking to it, and it reacts accurately enough that I suspect it's more sophisticated than "wait for silence." If there is a "notice the end of speaking" model - which is not a crazy assumption - then I can imagine a slightly more complicated model that notices a combination of "now is a good time to talk + I have something to say."
It's surprising people still consider large scale language models as a key solution to the problem of AGI, when it has become quite clear they will hit all practical scaling limits without surpassing the "well informed imbecile" intelligence threshold.
All evidence points towards human reason as a fundamentally different approach, orders of magnitude more efficient at integrating and making sense of ridiculously smaller amounts of training data.
My instinct is that this is probably on the naive side. For instance, we use separation of concerns in our systems because we're too cognitively limited to create and manage deeply integrated systems. Nature doesn't have that problem.
For instance, the idea that we can neatly have the emotion system separate from the motor control system. Emotions are a cacophony of chemicals and signals traversing the entire body - they're not an enum of happy/angry/sad - we just interpret them as such. So you probably don't get to isolate them off in a corner.
Basically I think it's very tempting to severely underestimate the complexity of a problem when we're still only in theory land.
Would recommend reading The Ravenous Brain: How the New Science of Consciousness Explains Our Insatiable Search for Meaning by Daniel Bor for a lot of ideas strongly connected to recent research. My interpretation of this is the mind ends up being a story processing machine that builds stories about what has happened and is happening and constructs and compares stories about what might happen or be made to happen. Of course it is difficult to summarize a whole book rich with references in a sentence, but the model seems arguably more simple and well established than what you are currently putting forward.
Very much looking forward to seeing continuing progress in all this.
> I’m motivated by the success of AI-based language models to look at the future of digital minds.
When intelligent machines are constructed, we should not be surprised to find them as confused and as stubborn as men in their convictions about mind-matter, consciousness, free will, and the like.
The model is good. Environment -> Perception -> Planning/Imagining -> Acting -> Learning from feedback.
What is missing from this picture is the social aspect. No agent got too smart alone, it's always an iterative "search and learn" process, distributed over many agents. Even AlphaZero had evolutionary selection and extensive self play against its variants.
Basically we can think of culture as compressed prior experience, or compressed search.
There's a ton missing from the article, and certain social training or skills are a big part of that.
Although it's not spelled out in the article, I'm hoping that the feature of agency along with an emotional system would enable constructive social behavior. Agency is helpful because it would empower AI models to meaningfully speak to each other, for example. Human emotions like empathy, social alignment, curiosity, or persistence could all help AI models to get along well with others.
The author talks about agency which require being able to independently take actions apart from reacting to an input. However, the feedback provided by a two-input model also limits the mind model as it now reacts to the feedback it receives when in listening mode. Isn't it contradictory to the concept if agency?
The idea of "agency" I have in mind is simply the option to take action at any point in time.
I think the contradiction you see is that the model would have to form a completion to the external input it receives. I'm suggesting that the model would have many inputs: one would be the typical input stream, just as LLMs see, but another would be its own internal recent vectors, akin to a recent stream of thought. A "mode" is not built in to the model; at each token point, it can output whatever vector it wants, and one choice is to output the special "<listening>" token, which means it's not talking. So the "mode" idea is a hoped-for emergent behavior.
Some more details on using two input streams:
All of the input vectors (internal + external), taken together, are available to work with. It may help to think in terms of the typical transformer architecture, where tokens mostly become a set of vectors, and the original order of the words are attached as positional information. In other words, transformers don't really see a list of words, but a set of vectors, and the position info of each token becomes a tag attached to each vector.
So it's not so hard to merge together two input streams. They can become one big set of vectors, still tagged with position information, but now also tagged as either "internal" or "external" for the source.
The model is interesting. This is similar in parts to what we are building at nonbios. So for example sensory inputs are not required to simulate a model of a mind. If a human cannot see, the human mind is still clearly human.
Model training seems to me to be much closer to simulating the evolution of the human mind starting from single cell bacteria, rather than the development of the mind of a baby up to a fully functional human. If so, then sensory inputs and interaction with the physical through them were absolutely a crucial part of how minds evolved, so I find your approach a priori very unlikely to have a chance at success.
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.
You’re on the right track :). Check out The Science of Logic, Neurophilosophy, I am A Strange Loop, Brainstorms, and Yudkowsky’s earlier work, if you haven’t! Based on what you have here, you’d love em. It’s a busy field, and a lively one IME. Sadly, the answer is no: the anxiety never goes away
Ever since LLM’s came out many of us has been wondering these things. It would be easy to say that perhaps our attention and senses somehow come together to formulate prompts and thoughts etc what appears in the mind is the output. And everything we ever experienced has trained the model.
But of course we can be assured it’s not quite like that in reality. This is just another example of how our models for explaining the life are reflection of the current technological state.
Nobody considers that old clockwork universe now, and these AI inspired ideas are going to fall short all the same. Yet, progress is happening and all these ideas and talks are probably important steps that carry us forward.
tylerneylon|1 year ago
Context for the article: I'm working on an ambitious long-term project to write a book about consciousness from a scientific and analytic (versus, say, a meditation-oriented) perspective. I didn't write this fact in the article, but what I'd love to happen is that I meet people with a similar optimistic perspective, and to learn and improve my communication skills via follow-up conversations.
If anyone is interested in chatting more about the topic of the article, please do email me. My email is in my HN profile. Thanks!
jcynix|1 year ago
Quoted from rom https://www.newyorker.com/magazine/2017/03/27/daniel-dennett...
Regarding the multiple layers: the most interesting thoughts I read about theories of a mind, are the books by Marvin Minsky, namely: The Society of Mind and The Emotion Machine which should be more widely known.
More on Minsky's ideas on “Matter, Mind, and Models” are mentioned in https://www.newyorker.com/magazine/1981/12/14/a-i
freilanzer|1 year ago
rerdavies|1 year ago
Pinker advocates for a model of mind that has multiple streams of consciousness. In Pinker's model of mind, there are multiple agents constructing models of the world in parallel, each trying to predict future states from current state plus current input data. A supervisory process then selects the model that has made the best prediction of current state in the recent past for use when reacting in the current moment. The supervisor process is free to switch between models on the fly as more data comes in.
Pinker grounds his model of mind in curious observations of what people remember, and how our short term memories change over a period of sometimes many seconds as our mind switches between different agent interpretations of what's going on. Witnesses are notoriously unreliable. Pinker concerns himself with why and how witnesses are unreliable, not for legal reasons, but for how those unreliabilities might reveal structure of mind. Pinker's most interesting observation (I think) is that what we seem to remember is output from the models rather than the raw input data; and that what we remember seeing can change dramatically over a period of many seconds as the supervisory process switches between models. Notably, we seem to remember, in short term memory, details of successful models that make "sense", even when those details contradict with what we actually saw. And when those details are moved to longer-term memory, the potentially inaccurate details of model are what are committed.
ypeterholmes|1 year ago
In particular, I think there's a nice overlap between my description of self-awareness and yours. You mention a model capable of seeing its own output, but I take it a step further and indicate model capable of generating its own simulated actions and results. Curious what you think, thanks!
lukasb|1 year ago
mensetmanusman|1 year ago
It will be a good practice to see how deeply terms can be applied in order to combat this gap.
FrancisMoodie|1 year ago
Animats|1 year ago
Too many people have written books about consciousness. There's much tail-chasing in that space, all the way back to Aristotle. Write one about common sense. Current AI sucks at common sense. We can't even achieve the level of common sense of a squirrel yet.
Working definition of common sense: getting through the next 30 seconds of life without a major screwup.
bubblyworld|1 year ago
There's a theory that real brains subvert this, and what we perceive is actually our internal model of our self/environment. The only data that makes it through from our sense organs is the difference between the two.
This kind of top-down processing is more efficient energy-wise but I wonder if it's deeper than that? You can view perception and action as two sides of the same coin - both are ways to modify your internal model to better fit the sensory signals you expect.
Anyway, I guess the point I'm making is you should be careful which way you point your arrows, and of designating a single aspect of a mind (the action centre) as fundamental. Reality might work very differently, and that maybe says something? I don't know haha.
gnz11|1 year ago
unknown|1 year ago
[deleted]
privacyonsec|1 year ago
ergonaught|1 year ago
paulmooreparks|1 year ago
Nevermark|1 year ago
It is just meta self-awareness.
We evolved to be aware, model & react and to opportunistically control our environment. Loop 1.
Then we evolved to be aware, model, & & react and to opportunistically control our selves as bodies. Loop 2.
Then we evolved to be aware, model, & & react and to opportunistically control those activities too, our minds. In the process of being able to model & react to ourselves modelling & reacting to ourselves we completed a self-awareness of self-awareness loop. Loop 3.
I think this is a good explanation because what else is consciousness but knowing you are conscious? Self-aware that you are self-aware?
And it makes perfect sense as an evolutionary trajectory using greater levels of neural representation of our practical relationship with our environment, including ourselves, to create higher level survival options.
So this definition is both a functional and developmental explanation of the emergent phenomenon of consciousness.
Just as we only have partial access to information and control of our environment, our bodies, we are also limited to the degree we are aware and can control our mind. Introspection.
The next level up is “theory of mind”, “empathy”, etc. our ability to model & interact with others mind’s as individuals and groups, reciprocally. Loop 4. That created society and culture, living information that extends outside us and grows and lives beyond each of us.
Infusing our thoughts, and progressively thinking abilities into technology, that in theory and not too distant practice could have access to all parts of their own minds’ states, operations and design, would be Loop 5. When deeply conscious beings start, things will get interesting in the Sol system.
When people start talking about quantum waves or smart rocks or universes in the context of consciousness I feel a little ill. People like to “solve” unrelated mysteries by merging them and shouting “tada!” (“We are puzzled about how they built the pyramids… but we don’t know if there are aliens either, so obviously… these mysteries solve each other!”)
cheesewheel|1 year ago
"I think, therefore I am"
ilaksh|1 year ago
Before that you can look into the AGI conference people like Ben Goertzel, Pei Wang. And actually the whole history of decades of AI research before it became about narrow AI.
I'd also like to suggest that creating something that truly closely simulates a living intelligent digital person is incredibly dangerous, stupid, and totally unnecessary. The reason I say that is because we already have superhuman capabilities in some ways, and the hardware, software and models are being improved rapidly. We are on track to have AI that is dozens if not hundreds of times faster than humans at thinking and much more capable.
If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
Don't get me wrong, I love AI and my whole life is planned around agents and AI. But I no longer believe it is wise to try to go all the way and create a "real" living digital species. And I know it's not necessary -- we can create effective AI agents without actually emulating life. We certainly don't need full autonomy, self preservation, real suffering, reproductive instincts, etc. But that is the goal he seems to be down in this article. I suggest leaving some of that out very deliberately.
doctor_eval|1 year ago
> it will actually out-compete us for resource control. And will no longer be a tool we can use.
I’ve never been convinced that this is true, but I just realised that perhaps it’s the humans in charge of the AI who we should actually be afraid of.
sonink|1 year ago
I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down.
AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.
Jensson|1 year ago
Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.
devodo|1 year ago
This is a very strong argument. Certainly all the ingredients to replicate a mind must exist within our physical reality.
But does an algorithm running on a computer have access to all the physics required?
For example, there are known physical phenomena, such as quantum entanglement, that are not possible to emulate with classical physics. How do we know our brains are not exploiting these, and possibly even yet unknown, physical phenomena?
An algorithm running on a classical computer is executing in a very different environment than a brain that is directly part of physical reality.
deepburner|1 year ago
QC researcher here, strictly speaking, this is false. Clifford circuits can be efficiently simulated classically and they exhibit entanglement. The bottom line is we're not entirely sure where the (purported) quantum speedups come from. It might have something to do with entanglement, but it's not enough by itself.
Re: about mermin's device, im not sure why you think it can not be simulated classically when all of the dynamics involved can be explained by 4x4 complex matrices.
clob|1 year ago
This is wrong. You can even get a mod for Minecraft which implements quantum mechanics.
https://www.curseforge.com/minecraft/mc-mods/qcraft-reimagin...
monocasa|1 year ago
https://en.wikipedia.org/wiki/Soar_%28cognitive_architecture...
tylerneylon|1 year ago
Jensson|1 year ago
How would it intelligently do this? What data would you train on? You don't have trillions words of text where humans wrote what they thought silently interwoven with what they wrote publicly.
History has shown over and over that hard coded ad hoc solutions to these "simple problems" never work to create intelligent agents, you need to train the model to do that from the start you can't patch in intelligence after the fact. Those additions can be useful, but they have never been intelligent.
Anyway, such a model I'd call "stream of mind model" rather than a language model, it would fundamentally solve many of the problems with current LLM where their thinking is reliant on the shape of the answer, while a stream of mind model would shape its thinking to fit the problem and then shape the formatting to fit the communication needs.
Such a model as this guy describes would be a massive step forward, so I agree with this, but it is way too expensive to train, not due to lack of compute but due to lack of data. And I don't see that data being done within the next decade if ever, humans don't really like writing down their hidden thoughts, and you'd need to pay them to generate data amounts equivalent to the internet...
tylerneylon|1 year ago
It's a fair question, and I don't have all the answers. But for this question, there might be training data available from everyday human conversations. For example, we could use a speech-to-text model that's able to distinguish speakers, and look for points where one person decided to start speaking (that would be training data for when to switch modes). Ideally, the speech-to-text model would be able to include text even when both people spoke at once (this would provide more realistic and complete training data).
I've noticed that the audio mode in ChatGPT's app is good at noticing when I'm done speaking to it, and it reacts accurately enough that I suspect it's more sophisticated than "wait for silence." If there is a "notice the end of speaking" model - which is not a crazy assumption - then I can imagine a slightly more complicated model that notices a combination of "now is a good time to talk + I have something to say."
cornholio|1 year ago
All evidence points towards human reason as a fundamentally different approach, orders of magnitude more efficient at integrating and making sense of ridiculously smaller amounts of training data.
abcde777666|1 year ago
For instance, the idea that we can neatly have the emotion system separate from the motor control system. Emotions are a cacophony of chemicals and signals traversing the entire body - they're not an enum of happy/angry/sad - we just interpret them as such. So you probably don't get to isolate them off in a corner.
Basically I think it's very tempting to severely underestimate the complexity of a problem when we're still only in theory land.
m0llusk|1 year ago
Very much looking forward to seeing continuing progress in all this.
whitten|1 year ago
https://kar.kent.ac.uk/21525/2/A_theory_of_the_acquisition_o...
Memory Organisation Packets might also deal with issues encountered.
https://www.cambridge.org/core/books/abs/dynamic-memory-revi...
jcynix|1 year ago
When intelligent machines are constructed, we should not be surprised to find them as confused and as stubborn as men in their convictions about mind-matter, consciousness, free will, and the like.
Minsky, as quoted in https://www.newyorker.com/magazine/1981/12/14/a-i
visarga|1 year ago
What is missing from this picture is the social aspect. No agent got too smart alone, it's always an iterative "search and learn" process, distributed over many agents. Even AlphaZero had evolutionary selection and extensive self play against its variants.
Basically we can think of culture as compressed prior experience, or compressed search.
tylerneylon|1 year ago
Although it's not spelled out in the article, I'm hoping that the feature of agency along with an emotional system would enable constructive social behavior. Agency is helpful because it would empower AI models to meaningfully speak to each other, for example. Human emotions like empathy, social alignment, curiosity, or persistence could all help AI models to get along well with others.
navigate8310|1 year ago
tylerneylon|1 year ago
I think the contradiction you see is that the model would have to form a completion to the external input it receives. I'm suggesting that the model would have many inputs: one would be the typical input stream, just as LLMs see, but another would be its own internal recent vectors, akin to a recent stream of thought. A "mode" is not built in to the model; at each token point, it can output whatever vector it wants, and one choice is to output the special "<listening>" token, which means it's not talking. So the "mode" idea is a hoped-for emergent behavior.
Some more details on using two input streams:
All of the input vectors (internal + external), taken together, are available to work with. It may help to think in terms of the typical transformer architecture, where tokens mostly become a set of vectors, and the original order of the words are attached as positional information. In other words, transformers don't really see a list of words, but a set of vectors, and the position info of each token becomes a tag attached to each vector.
So it's not so hard to merge together two input streams. They can become one big set of vectors, still tagged with position information, but now also tagged as either "internal" or "external" for the source.
freilanzer|1 year ago
rnewme|1 year ago
sonink|1 year ago
tsimionescu|1 year ago
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.
mensetmanusman|1 year ago
Simplicitas|1 year ago
bbor|1 year ago
miika|1 year ago
But of course we can be assured it’s not quite like that in reality. This is just another example of how our models for explaining the life are reflection of the current technological state.
Nobody considers that old clockwork universe now, and these AI inspired ideas are going to fall short all the same. Yet, progress is happening and all these ideas and talks are probably important steps that carry us forward.
0xWTF|1 year ago
antiquark|1 year ago