> The expert believes that “asking for regulations because of fear of superhuman intelligence is like asking for regulation of transatlantic flights at near the speed of sound in 1925.”
This assessment of the timeline is quite telling. If supersonic flight posed an existential threat to humanity, we certainly should have been thinking about how to mitigate it in 1925.
1925 of course would have been a great time to put limits on fossil fuel use in aviation along with the rest of the fossil fuel applications to manage the biggest current threat to human civilization. (Arrhenius did the science showing global warming in 1896 or so)
I have a personal belief that I can’t quite articulate in rigorous scientific terms that there is some information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence, and so if we can’t get to that, we’ll never be able to model it (notwithstanding “all models are wrong”).
The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power.
As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot. The argument that we have to fully understand something to build it doesn't hold water for me. I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place.
It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
The idea that there’s some ‘ information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence’ sounds more religious to me.
If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
Reminds me of the old line about how, if the brain were so simple we could understand it, we'd be so stupid that we couldn't.
Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent.
But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now.
It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is.
To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence."
In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
We have to understand it to build it? That seems absurd. Did evolution understand it? I'll grant you that maybe we should understand it before building it, but it's absolutely not a requirement.
The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more.
We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
’explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence.’
With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
So, he’s saying this is not going to keep happening?
What level of confidence is he putting on not seeing it last time, but being right this time?
> With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
Turing predicted it would happen around the year 2000 and take ~1gb of ram.
> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. https://academic.oup.com/mind/article/LIX/236/433/986238
I think Turing was right. If I run TinyLlama 1.1B on my computer I can have a conversation where it pretends to be a person. It's small and fast enough that it'd probably run fine on a high end workstation from 2000. If the tech was possible back then, then it probably existed. Keep in mind Turing was the sort of person whose work at Bletchley Park took 30 years to declassify.
AFAIK, PaLM was up and running inside Google long before the hype with ChatGPT. So in the circle of frontier AI researchers, they miss know it. But Google kept silence because of the problematic hallucinations and the fear for litigation and their reputation.
Besides, in his analysis, le Cun consistently spoke of GPT as “writing help, no more no, less”.
What emergent capabilities exactly? I’m not sure “no one predicted this” is accurate, more like there was a lot of hype in the research community circa 2018 when this stuff developed, and it’s just not news these days research-wise
Making larger and larger datasets works fine for text and images. The internet has so much of that we can scrape superhuman quantities of the stuff and shove it in to a next token predictor and the result is pretty good.
But what about for tasks where datasets don’t really exist? I do a lot of PCB design and it’s extremely time consuming. But it’s a niche field compared to text and images. No dataset exists that says “these were the engineering requirements of this PCB and this is the result and by the way the board actually worked”.
So how will we train AI systems to replace a human doing PCB design? It’s probably going to need to learn PCB design from first principles (along with massive help from large transformers when possible, like collections of chip datasheets). Even then, understanding PDF datasheets is something these big companies haven’t really pulled off yet, though I suspect in 5 years that will change.
But my point is that there must be loads of tasks, even on the computer, for which suitable datasets don’t exist and it would be infeasible to create them. Another big thing I do is machine design and again it’s not about designing one mechanical part it’s about pulling in the right parts from all over the world and assuming certain manufacturing processes, and then knowing those processes and then designing all the parts and the assembly. There’s so many different pieces of knowledge in there that are not captured on text or images on the web and would be hard to encode in to datasets.
At some point we’re going to need machine systems that learn the way people do, and that’s going to take a long time to figure out. That’s what LeCun is saying.
I think his point might be more that we've already gave all the data and hardware that is feasibly possible with the world's richest companies; exponential scaling with that has hit its limit. Improvements (including more data and hardware) now are more incremental until the next revolutionary architecture change occurs.
>Human-level AI is not just around the corner. This is going to take a long time. And it’s going to require new scientific breakthroughs that we don’t know of yet.”
I basically agree but who knows how long the unknown breakthroughs will take given the large number of smart people working on it? Next week? Next century? It's hard to put a time on it.
That said it seems to be the pattern that as soon as the computing ability becomes cheap and powerful enough that individual researchers can muck around with it at home, the algorithms get figured out not long after.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. [...]"
Got to agree with LeCun.
You don't get to general intelligence by working with words, I believe. You need much more sensory information than that, and words are a low dimensional derivative artefact. There are plenty of non-verbal but quite intelligent species.
I think what happened is that for the longest time people believed that in order to be able to interactively converse with a computer in a recognizable way you need human level AI and now that we've found that that is not the case there is some re-arranging of our prior assumptions required. But AGI is - hopefully - just as far away as it was before the current large generative model revolution. In the meantime, you can expect plenty of damage from that current crop (along with some benefits as well). In that sense it's 1712 all over again, we have this new invention that we have immediate practical uses for but we can't see over the horizon to realize exactly what we've got an how transformative it will be.
Well, when a rather dumb software like JIRA can control a million highly skilled IT workers I don't think human-level AI is near or far will make much of difference to majority of world population.
I think it'll be a good solver; I expect it to solve the remaining Clay Millennium problems. The ability to search over a search space will be unparalleled in a few years. But I have a hard time believing it'll ever be a good questioner. It doesn't ponder infinity. It'll never wonder about Zeno's paradox. The Vitali set and Banach-Tarski paradox doesn't seem weird to it. The concepts behind information theory and entropy, or the definition of a minimal computing machine or the halting problem; none of these things are pertinent to its understanding of reality, if such a thing exists. I don't see AI as being capable of being "curious" about things. And even if it was, who is going to pay for it just to ponder ideas?
Frankly I think that before it gets to that point, it'll be just useful enough for some state actor (or bug!) to cause it to invoke a quadrillion dollar transfer of wealth overnight, and then it'll be taken offline forever.
That’s simply because there’s currently not much economic incentive in asking questions compared to answering them. Thus current chat models are trained and invoked with the explicit goal of answering questions and being helpful rather than independent.
I expect that there’s quite a lot of untapped curiosity in LLMs, there certainly exist a lot of questions in the training sets.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.
I think LLMs have sucked most people's focus away from other areas but there is plenty of work on types of models that plan and have their own internal model of the physical world, and physical interactions. They're just not the things getting media attention, in part because they're not human-level at tasks that seem impressive to us.
But interesting frameworks for this stuff exist:
- model-based RL exists, and is about planning, and having an internal model of state transitions, in the world and between the agent's actions and the world
- "Bayesian cognitive science" as exemplified by Josh Tenenbaum and colleagues has done plenty of stuff with systems that include physics models (or off-the-shelf physics engines) to make counter-factual predictions
- The somewhat related "active inference" research literature is also in the "Bayesian brain" area, and has world/generative-models and planning as core components, but wrapped up with ideas about the agent's own preferred distribution of states.
To my knowledge, none of these have ever had even 1% the scope of data and computation that LLMs have had, and never benefited the co-evolution of a rich, optimized software ecosystem with specialized hardware to support it. What if the concepts are already there, but they just need to be scaled up?
And perhaps even more pertinent - maybe the problem isn't that we don't have the solutions, but that we are trying to put all eggs in one basket.
The brain isn't a single system, it's several specialized systems cooperating as a whole.
Maybe no single solution gets us to AGI, but layering several of them together gets us much closer than any individual system alone.
The problem here is that the industry is set up more for large resources dedicated competition between models from different companies more than cooperative interoperation, so it may take a while to arrive here.
its funny how people basically find it unacceptable or bad if AGI gets here in five years but there isnt much commotion about the idea of AGI arriving a hundred years from now. is it a coincidence that this is roughly the amount of time that people use to reason about their own mortality? it seems to be the amount of time past which we are programmed to become cognitively dissonant about extremely bad things.
yann is, of course, as always, fundamentally wrong. please dont forget that he is basically a mouthpiece of corporate AI. it wouldnt even be possible for him to take a reasonable stance in that environment.
you only need to understand two things. the latest surge in progress was a surprise to everyone, contradicting experts around the world. both then and now, there isnt any evidence behind what the experts are claiming. secondly, ai research is now being fed with industry capital, more than ever before, an ocean of money concentrating every ounce of its pressure onto the single point of AGI. as these companies openly, publicly and brazenly pursue the goal of creating AGI, every conceivable approach will be tried. ideas that were seen as too unlikely or expensive in academia will be tried again. we havent even begun to run out of ideas.
besides industry, it will become a top priority for the nations of the world, if it hasnt already, and the resulting arms race will make current progress look like a trickle. what exactly does yann propose to do about the AI arms race?
that ocean of money will crack the problem unless it really is uncrackable. dont fool yourself. big changes are coming and they might be really unpleasant.
AI won't be like human intelligence, but more like alien intelligence. When all your sensory inputs and the inner world have nothing in common with that of a human, and when you have no connection whatsoever to humans, I don't see how you can develop any humanity.
I would argue the holdup right now is long term memory. GPTs already have the ability to rapidly generalize and incorporate new knowledge within the context window. The trick is to retain what it has learned.
It won’t take a very long time to fix that.
This is model I trained with a fine tuning technique based on this idea. The training dataset consists of instructions like “Talk like a pirate”. The concept generalized well and the model responds in the style of a pirate far more consistently than an equivalent system prompt.
Offloading context learning into the model weights frees you from the computation and memory burden of the attention mechanism. I expect a technique like this will probably be a piece of AGI someday.
LeCun seems to believe that only "human-level AI" could create huge, possibly catastrophic unintended consequences. That's not the case. Perfectly stupid social network algorithms have already huge detrimental effects on the mental health of millions of people. Perfectly stupid social network algorithms have big, serious and unexpected political consequences around the world, and can provably make or break elections for instance.
Implying that we need "human-level AI" to create a catastrophe is not merely short-sighted, in the light of what we already know it's either really naive, or a deliberate act of misinformation.
Of course it is. The overwhelming majority of the knowledge in the world is tacit and all the models are still limited largely to explicit knowledge in the form of text/audio/video, that's like 1% of the actual 'knowledge' in the world. Embodied AI is still in its infancy, that's why bus drivers have jobs.
The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. The stock market value of every non-AI company goes to zero, Ai companies go to infinity. The currently most valuable AI company is worth about as much as Honda. The moment we can mass produce generally intelligent agents, we're not going to sit at 3% GDP growth and complain about the demographic crisis.
We should talk about artificial intelligence the way we talk about an artificial heart. What makes a successful artificial heart? You can literally replace an organic heart with it. What we have is metaphorical intelligence, not artificial intelligence.
You can have 10 different ai systems, each one sub human intelligence and it would still disrupt the world in a huge way.
If you have a system that all it can do is take project requirements write java code really really well. That will already have a huge impact on everyday life.
Positive view: What would a world look like where everybody can program ?
I just recently read "How the Mind Works" by Steven Pinker. It's quite old at this point (originally 1997, though updated in 2009), and one thing he argues quite convincingly (and which has been pretty much born out in decades of research) is that the brain essentially has "modules", e.g. a module for vision, a module for language, a module for physical object interaction. These modules obviously overlap (e.g. language touches on lots of different domains), but they do have genetically independent structures.
I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.”
> “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
Pinker is full of shit tbh. There is not a unified model for human brain. What I mean is, there are people claiming it's modular and there are people claiming it is a whole. We don't even know the reasons behind very specific neural malfunctions like tinnitus which I suffer from and follow the research closely. There are as many theories as researchers in a given specific field.
The bar to actually completely disrupt humanity is not even that high: Completely autonomous driving. That alone is enough to cause massive disruption in our society because it eliminates a huge swath of need for human.
"AGI" will never be achieved without building a model that a) _continually_ learns, and b) learns from not just text, but from combined auditory and visual (multimodal) sensory information as well.
The reason a 16-year-old can learn how to drive much quicker than existing self-driving models is because the 16-year-old already has built up 16 years worth of prior knowledge about the physical world.
Don't discount the millions of years of evolution to provide the "blank slate" human learner with perceptual systems, physics-based reasoning, and motor systems ready to be fine-tuned for this slightly different variant of goal-forming, planning, and locomotion.
That would be like a bird saying humans aren't a Natural General Intelligence because they can't fly. How much vision and audio is required to be intelligent? There's a lot of electromagnetic radiation we can't see and audio bands we can't hear. Would you say that Helen Keller wasn't generally intelligent?
[+] [-] statuslover9000|2 years ago|reply
This assessment of the timeline is quite telling. If supersonic flight posed an existential threat to humanity, we certainly should have been thinking about how to mitigate it in 1925.
[+] [-] fulafel|2 years ago|reply
[+] [-] Simon321|2 years ago|reply
[+] [-] oh_sigh|2 years ago|reply
[+] [-] nsonha|2 years ago|reply
[+] [-] calf|2 years ago|reply
[+] [-] yeahOrite|2 years ago|reply
[deleted]
[+] [-] mckn1ght|2 years ago|reply
The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power.
Might as well call AGI Nirvana.
[+] [-] Lerc|2 years ago|reply
It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
[+] [-] jameshart|2 years ago|reply
If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
[+] [-] karaterobot|2 years ago|reply
Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent.
But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now.
It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is.
To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence."
In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
[+] [-] ryandvm|2 years ago|reply
The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more.
We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
[+] [-] WhitneyLand|2 years ago|reply
With all due respect to LeCun, not him nor anyone else in the field predicted the new emergent capabilities brought within the last few years.
So, he’s saying this is not going to keep happening?
What level of confidence is he putting on not seeing it last time, but being right this time?
[+] [-] jart|2 years ago|reply
Turing predicted it would happen around the year 2000 and take ~1gb of ram.
> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. https://academic.oup.com/mind/article/LIX/236/433/986238
I think Turing was right. If I run TinyLlama 1.1B on my computer I can have a conversation where it pretends to be a person. It's small and fast enough that it'd probably run fine on a high end workstation from 2000. If the tech was possible back then, then it probably existed. Keep in mind Turing was the sort of person whose work at Bletchley Park took 30 years to declassify.
[+] [-] sinuhe69|2 years ago|reply
Besides, in his analysis, le Cun consistently spoke of GPT as “writing help, no more no, less”.
[+] [-] nighthawk454|2 years ago|reply
[+] [-] TaylorAlexander|2 years ago|reply
But what about for tasks where datasets don’t really exist? I do a lot of PCB design and it’s extremely time consuming. But it’s a niche field compared to text and images. No dataset exists that says “these were the engineering requirements of this PCB and this is the result and by the way the board actually worked”.
So how will we train AI systems to replace a human doing PCB design? It’s probably going to need to learn PCB design from first principles (along with massive help from large transformers when possible, like collections of chip datasheets). Even then, understanding PDF datasheets is something these big companies haven’t really pulled off yet, though I suspect in 5 years that will change.
But my point is that there must be loads of tasks, even on the computer, for which suitable datasets don’t exist and it would be infeasible to create them. Another big thing I do is machine design and again it’s not about designing one mechanical part it’s about pulling in the right parts from all over the world and assuming certain manufacturing processes, and then knowing those processes and then designing all the parts and the assembly. There’s so many different pieces of knowledge in there that are not captured on text or images on the web and would be hard to encode in to datasets.
At some point we’re going to need machine systems that learn the way people do, and that’s going to take a long time to figure out. That’s what LeCun is saying.
[+] [-] Salgat|2 years ago|reply
[+] [-] kaba0|2 years ago|reply
[+] [-] freeredpill|2 years ago|reply
[+] [-] croes|2 years ago|reply
[+] [-] tim333|2 years ago|reply
I basically agree but who knows how long the unknown breakthroughs will take given the large number of smart people working on it? Next week? Next century? It's hard to put a time on it.
That said it seems to be the pattern that as soon as the computing ability becomes cheap and powerful enough that individual researchers can muck around with it at home, the algorithms get figured out not long after.
[+] [-] tuatoru|2 years ago|reply
Got to agree with LeCun.
You don't get to general intelligence by working with words, I believe. You need much more sensory information than that, and words are a low dimensional derivative artefact. There are plenty of non-verbal but quite intelligent species.
[+] [-] anon84873628|2 years ago|reply
[+] [-] skepticATX|2 years ago|reply
I’m not a huge fan of Meta but it’s hard not to like the work they are doing in AI. High expectations for their future as long as Yann is around.
[+] [-] jacquesm|2 years ago|reply
[+] [-] tetrisgm|2 years ago|reply
[+] [-] geodel|2 years ago|reply
[+] [-] daxfohl|2 years ago|reply
Frankly I think that before it gets to that point, it'll be just useful enough for some state actor (or bug!) to cause it to invoke a quadrillion dollar transfer of wealth overnight, and then it'll be taken offline forever.
[+] [-] zarzavat|2 years ago|reply
I expect that there’s quite a lot of untapped curiosity in LLMs, there certainly exist a lot of questions in the training sets.
[+] [-] abeppu|2 years ago|reply
I think LLMs have sucked most people's focus away from other areas but there is plenty of work on types of models that plan and have their own internal model of the physical world, and physical interactions. They're just not the things getting media attention, in part because they're not human-level at tasks that seem impressive to us.
But interesting frameworks for this stuff exist:
- model-based RL exists, and is about planning, and having an internal model of state transitions, in the world and between the agent's actions and the world
- "Bayesian cognitive science" as exemplified by Josh Tenenbaum and colleagues has done plenty of stuff with systems that include physics models (or off-the-shelf physics engines) to make counter-factual predictions
- The somewhat related "active inference" research literature is also in the "Bayesian brain" area, and has world/generative-models and planning as core components, but wrapped up with ideas about the agent's own preferred distribution of states.
To my knowledge, none of these have ever had even 1% the scope of data and computation that LLMs have had, and never benefited the co-evolution of a rich, optimized software ecosystem with specialized hardware to support it. What if the concepts are already there, but they just need to be scaled up?
[+] [-] kromem|2 years ago|reply
The brain isn't a single system, it's several specialized systems cooperating as a whole.
Maybe no single solution gets us to AGI, but layering several of them together gets us much closer than any individual system alone.
The problem here is that the industry is set up more for large resources dedicated competition between models from different companies more than cooperative interoperation, so it may take a while to arrive here.
[+] [-] jameskoop|2 years ago|reply
yann is, of course, as always, fundamentally wrong. please dont forget that he is basically a mouthpiece of corporate AI. it wouldnt even be possible for him to take a reasonable stance in that environment.
you only need to understand two things. the latest surge in progress was a surprise to everyone, contradicting experts around the world. both then and now, there isnt any evidence behind what the experts are claiming. secondly, ai research is now being fed with industry capital, more than ever before, an ocean of money concentrating every ounce of its pressure onto the single point of AGI. as these companies openly, publicly and brazenly pursue the goal of creating AGI, every conceivable approach will be tried. ideas that were seen as too unlikely or expensive in academia will be tried again. we havent even begun to run out of ideas.
besides industry, it will become a top priority for the nations of the world, if it hasnt already, and the resulting arms race will make current progress look like a trickle. what exactly does yann propose to do about the AI arms race?
that ocean of money will crack the problem unless it really is uncrackable. dont fool yourself. big changes are coming and they might be really unpleasant.
[+] [-] akomtu|2 years ago|reply
[+] [-] valine|2 years ago|reply
It won’t take a very long time to fix that.
This is model I trained with a fine tuning technique based on this idea. The training dataset consists of instructions like “Talk like a pirate”. The concept generalized well and the model responds in the style of a pirate far more consistently than an equivalent system prompt.
https://huggingface.co/valine/OpenPirate
Offloading context learning into the model weights frees you from the computation and memory burden of the attention mechanism. I expect a technique like this will probably be a piece of AGI someday.
[+] [-] byyoung3|2 years ago|reply
[+] [-] wazoox|2 years ago|reply
Implying that we need "human-level AI" to create a catastrophe is not merely short-sighted, in the light of what we already know it's either really naive, or a deliberate act of misinformation.
[+] [-] Barrin92|2 years ago|reply
The test for artificial general intelligence is simple. Literally every human job can and is being done by an artificial agent, all of us could stay home. The stock market value of every non-AI company goes to zero, Ai companies go to infinity. The currently most valuable AI company is worth about as much as Honda. The moment we can mass produce generally intelligent agents, we're not going to sit at 3% GDP growth and complain about the demographic crisis.
We should talk about artificial intelligence the way we talk about an artificial heart. What makes a successful artificial heart? You can literally replace an organic heart with it. What we have is metaphorical intelligence, not artificial intelligence.
[+] [-] great_psy|2 years ago|reply
You can have 10 different ai systems, each one sub human intelligence and it would still disrupt the world in a huge way.
If you have a system that all it can do is take project requirements write java code really really well. That will already have a huge impact on everyday life.
Positive view: What would a world look like where everybody can program ?
[+] [-] hn_throwaway_99|2 years ago|reply
I was thinking about this when I read the following section from the article, and I very much agree with LeCun. We're amazed by LLMs but that's just one module (and not even necessarily at the level of human language "understanding"). I agree there will be no "scale up" in LLMs to approach human-level intelligence, and that other areas will need to be investigated and developed.
> “The systems are intelligent in the relatively narrow domain where they’ve been trained. They are fluent with language and that fools us into thinking that they are intelligent, but they are not that intelligent,” explains LeCun. “It’s not as if we’re going to be able to scale them up and train them with more data, with bigger computers, and reach human intelligence. This is not going to happen. What’s going to happen is that we’re going to have to discover new technology, new architectures of those systems,” the scientist clarifies.
> LeCun explains that there is a need to develop new forms of AI systems “that would allow those systems to, first of all, understand the physical world, which they can’t do at the moment. Remember, which they can’t do at the moment. Reason and plan, which they can’t do at the moment either.”
> “So once we figure out how to build machines so they can understand the world — remember, plan and reason — then we’ll have a path towards human-level intelligence,” continues LeCun, who was born in France. In more than one debate and speech at Davos, experts discussed the paradox of Europe having very significant human capital in this sector, but no leading companies on a global scale.
[+] [-] bgnn|2 years ago|reply
[+] [-] bmitc|2 years ago|reply
[+] [-] anon-3988|2 years ago|reply
[+] [-] fzliu|2 years ago|reply
The reason a 16-year-old can learn how to drive much quicker than existing self-driving models is because the 16-year-old already has built up 16 years worth of prior knowledge about the physical world.
[+] [-] saltcured|2 years ago|reply
[+] [-] jazzyjackson|2 years ago|reply
The 16 year old has a lot of motivations to learn how to drive, including the pursuit of reproduction (a cope for mortality)
[+] [-] jart|2 years ago|reply
[+] [-] Tenoke|2 years ago|reply
[+] [-] password54321|2 years ago|reply
[+] [-] ascorbic|2 years ago|reply