I replied to LeCun's claims about their latest protein structure predictor and he immediately got defensive. The problem is that i'm an expert in that realm and he is not. My statements were factual (pointing out real limitations in their system along with the lack of improvement over AlphaFold) and he responded by regurgitating the same misleading claims everybody in ML who doesn't understand biology makes. I've seen this pattern repeatedly.
It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
Same thing with making wildly optimistic claims about "obsoleting human radiologists in five years", made more than five years ago by another AI bigwig Geoffrey Hinton. They are doubtless brilliant researchers in their field, but they seem to view AI as a sort of cheat code to skip the stage where you actually have to understand the first thing about the problem domain to which it is applied, before getting to the "predictions about where the field is going".
Very similar to crypto evangelists boldly proclaiming the world of finance as obsolete. Rumours of you understanding how the financial system works were greatly exaggerated, my dudes.
I feel we are at crypto/blockchain levelof hype int ML and basically the old saying of "if you are a hammer, everything looks like a nail" applies.
For someone who dedicated their career to ML, they'll naturally try to solve everything in that framework. I observe this in every discipline that falls prey to it's own success. If there's a problem, those in the industry will naturally try to solve it with ML, often completely ignoring practical considerations.
Is the engine in your car underperforming? Let's apply ML. Has your kid bruised their knee while skating? Apply ML to his skating patterns.
The one saving grace of ML is that there are genuinely useful applications among the morass.
Without even opening the link I half expected it to be about LeCun and I want wrong.
Him and Grady Booch recently had a back and forth on the same subject on Twitter where to me it seemed like he couldn’t answer Booch’s very basic questions. It’s interesting to see another person with a similar opinion.
> It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
For sure. If this is how he treats outside experts, I can't imagine what it's like to work for him. Or rather, I can imagine it, and I think it does a lot to explain the release-and-panicked-rollback pattern.
ML people are the ultimate generalists. They claim to make tools which are domain agnostic, but they can't really validate that for themselves because they have no domain knowledge about anything.
Could you share the critical feedback you gave? I am interested as someone who works with biological systems and is curious about how ML can or cannot help.
I think it makes sense when you realize that the product (Galactica) and all the messaging around it are just PR - they're communicating to shareholders of a company in deep decline trying to say 'look at the new stuff we're doing, the potential for new business here'.
You interrupting the messaging ruins it, so you get some deniable boilerplate response. its not personal.
But we gave the keys to the economy to some vain children who have never had to do real work to make a name for themselves. Straight from uni being librarians assistants to the elders, straight to running the world!
Society is still led by vague mental imagery and promises of forever human prosperity. The engineering is right but no one asks if rockets to Mars are the right output to preserve consciousness. We literally just picked it up because the elders started there and later came to control where the capital goes.
We’re shorting so many other viable experiments to empower same old retailers and rockets to nowhere.
I'm delighted you called out these problems when you came across them, and sorry that he didn't have the grace or maturity to take it on board without getting defensive.
Like many thin-skinned hype merchants with a seven-figure salary to protect, they're going to try and block criticism in case it hits them in the pocket. Simple skin in the game reflex that will only hurt any chances of improvement.
It's tough because I think he has a really difficult job in many regards; Meta catches so much unjustified flak (in addition to justified flak) that being a figurehead for a project like this has to be exhausting.
Being constantly sniped at probably puts you in a default-unreceptive state, which makes you unable to take on valid feedback, as yours sounds like.
At some level he must know (AI) winter is coming along with the recession, which is why he is so defensive, as if a barrage of words will stave off the inevitable.
As a bright-eyed science undergraduate, I went to my first conference thinking how amazing it would be to have all these accomplished and intelligent people in my field all coming together to share their knowledge and make the world a better place.
And my expectations were exceeded by the first speaker. I couldn't wait for 3 full days of this! Then the second speaker got up, and spent his entire presentation telling why the first person was an idiot and totally wrong and his research was garbage because his was better. That's how I found out my field of study was broken into two warring factions, who spent the rest of the conference arguing with each other.
I left the conference somewhat disillusioned, having learned the important life lesson that just because you're a scientist doesn't mean you aren't also a human, with all the lovely human characteristics that entails. And compared to this fellow, the amount of money and fame at stake in my tiny field was miniscule. I can only imagine the kinds of egos you see at play among the scientists in this article.
I was working for somebody once who seemed to think LeCun was an uninspired grind and I'm like no, LeCun won a contest to make a handwritten digit recognizer for the post office. LeCun wrote a review paper on text classification that got me started building successful text classifiers and still influences my practice. LeCun is one of the few academics who I feel almost personally taught me how to do something challenging.
But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
Let's hope "longtermism" and other A.I. hype goes the way of "Web3".
AI Safety is important, but the unsafety isn't from the superintelligrnt AI, it's from dumb and cruel people hiding behind it as an excuse for their misbehavior.
The most aggrivating thing about EA "longtermism" AI Safety stuff is that is takes the oxygen out of the room for actual AI safety research.
Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want.
Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want.
Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want.
Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want.
Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want.
Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.
> But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous.
It boggles my mind how anyone can think otherwise. Existential dangers of superintelligent or even non-intelligent AI are the long-term result of the dangers of AI being developed and misused over time for human ends.
It's the exact same argument behind why we should be trying to track asteroids, or why we should be trying to tackle climate change: the worst-case scenario is unlikely or in the future, but the path we're on has numerous dangers where suffering and loss of human life is virtually certain unless something is done.
>But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
I hesitate to say "safe space", but... what if a group of people wants to come together discuss AI safety? If they'd have to regurgitate all the arguments and assumptions for everyone who comes along they'd never get anything done. If you are really interested to know where they are coming from, you can read the introductory materials that already exist. If the 99.9% of the world is hostile towards discussing AI safety (of the superintelligence explosion kind, not the corporate moralitywashing kind) there is some value in a place which is hostile to not discussing it, so that at least those interested can actually discuss it.
> it is hard to find a place where independent thought is so unwelcome.
Is that actually true, though? It's true that a higher fraction of the people in that community give credence to the intelligence explosion hypothesis than pretty much anywhere else. (This is what one would expect, since part of the purpose of LessWrong is to be a forum for discussions about super-intelligent AI.) But even if the intelligence explosion is a terrible, absolutely-wrong theory, that doesn't prevent the people who hold it from being open-minded and tolerant of independent thought. Willingness to consider new and different ideas is something the LessWrong community claims to value, so it would be a little bit weird if they were doing way worse than average at it. And AFAICT, it seems like they're doing fine. Some examples:
- Here [1] is a post critical of the intelligence explosion theory. It has 81 upvotes as of this writing, and the highest upvoted comment goes like: "thanks for writing this post, it makes a lot of good arguments. I agree with these things you wrote" (list of things) "here are some points where I disagree" (list of things). This may even be the original post you were talking about in your comment, except that it doesn't start with an apology.
- LW has 2 different kinds of voting: "Regular upvotes" provide an indication of the quality of a post or comment, and "agree/disagree votes" let people express how much they agree or disagree with a particular comment. Down-voting a high quality comment just because you disagree (instead of giving it a disagree-vote) would be against the culture on LW.
If you're already sure that LW is wrong about superintelligence, and you're trying to explain how they became wrong, then "those LW people were too open minded and fell for that intelligence explosion BS" makes more sense to me than anything about suppression of independent thought.
> "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous
Well that's an interesting way to misrepresent an entire important field of research based on what a few idiots said. There are serious people in that field who aren't addicted to posting on LessWrong.
I studied machine learning at NYU, and from interacting with Yann LeCun, I can say he’s actually a nice guy. Yes, his tweet is grumpy. I still feel as if the implication that Galactica should have been taken down was the worse thing happening here.
I read the MIT Technology Review article, and I was asking myself “what is an example of Galactica making a mistake?” The article could easily have quoted a specific prompt, but doesn’t. It says the model makes mistakes in terms of understanding what’s real/correct or not, but the only concrete example I see in the article is that the model will write about the history of bears in space with the implication that it’s making things up (and I believe the model does make such mistakes). I don’t think it’s a good article because it’s heavy on quoting people who don’t like the work and light on concrete details.
Does the imperfection of a language model really mean the model should not exist? This seems to be what some critics are aiming for.
I kinda agree with LeCun here. Why can't companies and people just put out cool things that have faults? Now we have a tool that got pulled, not because any concrete harm, only outrage over theoretical harm. It is not the tool, not the people finding faults, but people reaction's that seem like they have gone too far.
> In the company’s words, Galactica “can summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".
>Why can't companies and people just put out cool things that have faults?
Absolutely they can, and his employer could have kept it up. The issue is the phantasmagorical and ridiculous claims about AI-generated scientific research that LeCun peddles. When there's something concrete one can use to test these extremely bold claims, there's a way to at least partially apply a reality check to the claims, and demonstrate their ridiculousness. Which is a very useful and important part of how the scientific field evolves and advances. Feeding non-experts all these wild claims in perpetual future tense only works for so long, and it ought to be that way.
If you put something online, and present it as useful tool, then you have to expect that people are going to try to break it. You can look at that as free testing and open-source bughunting, or you can complain about about misuse and take it offline. The responsible parties took the latter route, which is kind of silly.
What this project created was something sophisticated and powerful, but not something people wanted, and they got (rightfully) pilloried for it. Instead of shaking ones fist at the world for rejecting your brilliance, maybe the really smart ones are making the things that others actually desire, and not merely developing techs that give themselves leverage over others and expecting the world to defer to this demonstration of intellectual prowess.
This whole incident was a case study for product management and startup school 101. I've made this exact same category of error in developing products, where I said, "hey, look at this thing I built that may mean you don't have to do what you do anymore!" and then was surprised when people picked it apart for "dumb" reasons that ignored the elegance of having automated some problem away.
If this model were really good, they would have used it to advance a bunch of new ideas in different disciplines before exposing it to the internet. Reality is, working at Meta/Facebook means they are too disconnected from the world they have influenced so heavily to be able to interpret real desire from people who live in it anymore. When you are making products to respond to data and no actual physical customer muse, you're pushing on a rope. I'd suggest the company has reached a stage of being post-product, where all that is left are "solutions," to the institutional customers who want some kind of leverage over their userbase, but no true source of human desire.
> LeCun also approvingly links to someone else who writes, in response to AI critic Gary Marcus
The article really fails to explain that LeCun and Marcus have been trading insults for the last few years, it's hardly LeCun snapping at some random person.
I'm not super familiar with the state of the art technology in this space and how these demos were presented, but I think all of these conflicts seem like they should be resolved if companies just put gigantic honking disclaimers on the work these AI tools produce.
If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."
Please correct me if I am misrepresenting a chain of events here.
A.. tool lands that allows one use language model to generate content. People feed it false data and share that, surprise, data it produces from the data the model is based on is false. How is this a surprise? I am still not sure why Meta would pull it? It can still be useful, but it was made not useful. I am not sure what a proper metaphor is for it, but it is almost like I give you a tool ( lets say a knife ) and you complain that the tool produces bad results when drinking soup.
What am I missing here?
<<or maybe it [Galactica] was removed because people like you [Marcus] abused the model and misrepresented it. Thanks for getting a useful and interesting public demo removed, this is why we can’t have nice things.
<<Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts.
Yann LeCun has lots of faults, certainly with how he treats AI safety in general, but a lot of the criticism he got was saying "He's not qualified to be in the position he is in" which is actually absurd if you know anything about him. Even if you knew nothing other than the fact that he won a Turing Award then he would be qualified for basically any computing/AI/ML job on the planet. \
Also the title of this post is deliberately inflammatory. Should be more like "Head of team that spent months building complex ML system annoyed when people spend undue amounts of time criticizing it."
I think there approach and model are interesting the problem is that they overhyped it to the point that it would be unacceptable academically.
Their abstract says "In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
If I was a reviewer of this paper I would ask them to add (if they haven't so) significant section to the body of the paper highlighting the limitation of the model and the ways it can be misused. Including showing examples of wrong output.
I would then ask them to rewrite the abstract to include something along the lines "We also highlight the limitations of the model including inability to distinguish fact from fiction in several instances and the ways it can be misused and outline some ideas on how these limitation could be mitigated or overcome in the future."
A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text
Isn't this the same problem that Github Copilot has?
Fundamentally it has no idea whether code works. It doesn't even know what the problem is.
It just spits out things that are similar to things its seen before, including buggy code from Github repositories.
Not sure why it's so popular. I guess it helps you write status quo code faster (the status quo being buggy and slow) -- I would rather it help us write better code.
> and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light. It’s easy to spot fiction when it involves space bears, but harder with a subject users may not know much about.
Well, that matches our current experience with human-written Wikipedia articles pretty closely then.
The title of this post should tell you everything you need to know about its bias against LeCun, but both sides are in the wrong here. Meta shouldn't be over-hyping tools that have serious problems and the anti AI bunch need to stop beating a dead horse by only judging AI's progress soley by how it can be abused.
This is a good title because it succinctly captures the issue: LeCun hyped this work by making wildly inaccurate claims and cherry picking model outputs. Go read his original tweets about the model's capabilities. Read Facebook's own characterization of what this model could achieve.
Not only did they exaggerate and hype, but they also didn't even try to solve some of the most glaring issues. The efforts on toxicity mentioned in their paper aren't even mid. They barely put effort into measuring the issue, and definitely didn't make any attempt to mitigate or correct.
Toxicity isn't really the point. Here's the point. If you can't prevent a model from being overtly toxic, then why should I believe you can give any guarantee at all about the model's output? I shouldn't, because you can't.
Galactica is just another a language model. It can be a useful tool. Facebook and LeCun oversold its capabilities and downplayed its issues. If they had just been honest and humble, things would've probably gone very differently.
In some sense, this is good news. The deep learning community -- and generative model work in particular -- is getting a much-needed helping of humble pie.
Hopefully we can continue publishing and hosting models without succumbing to moral panic. But the first step toward that goal is for scientists to be honest about the capabilities and limitations of their models.
----
My account is new so I am rate limited and unable to reply to replies. My response to the general vibes of replies is therefore added to the above post as an edit. Sorry.
Response about toxicitiy:
It's a proxy that they say they care about. I can stop there, but I'll also point out: it's not just "being nice", it's also stuff like overt defense of genocide, instructions for making bombs, etc. These are lines that no company wants their model to cross, and reasonably so. If you can't even protect Meta enough to keep the model online for more than a day or two, then why should I believe you can give any guarantee at all about the model's output in my use case? (And, again, they can't. It's a huge problem with LLLMs)
Response about taking the model down:
I'm not at FB/Meta, but I think I know what happened here.
In the best case, Meta was spending a lot of valuable zero-sum resources (top of the line GPUs) hosting the model. In the worst case they were setting a small fortune on fire at a cloud provider. Even at the largest companies with the most compute, there is internal competition and rationing for the types of GPUs you would need to host a Galactica-sized model. Especially in prototype phase.
An executive decided they would rather pull the plug on model hosting than spend zero-sum resources on a public relations snafu with no clear path to revenue. It was a business decision. The criticism of Galactica and especially the messaging around it was totally fair. The business decision was rational. Welcome to private sector R&D; it works a little different from your academic lab for better and for worse.
I came here just to ask: Could someone rewrite the HN title of this post? Currently it feels really clickbait-y.
One of the things I really like about HN is the _lack_ of clickbait titles. Some titles are more informative, some less, but overall I feel like the titles are clear, to the point, and not carefully crafted/engineered to poke the lizard part of my brain in the way that clickbait titles are.
Disclaimer: I haven't read the article so I can't propose a title myself. And with a title like this I'm not going to.
I have to agree. This is not my bailiwick, and, for all I know, the title is accurate, but it is jarring. I feel as if this is a professional venue, so I try to behave professionally (even though I'm not really looking for any work).
Gary Marcus built almost entirely his public reputation (which is positively correlated with his income) by antagonizing whatever Deep Learning scientist he could reach. He speaks badly about people that worked hard with their hands, brains and souls to make incredibly complex things happen.
Yann Lecun, which I personally met a couple of times, is in a way another sort of typical character: the ever-childish researcher that likes money a lot, to the point of accepting a prestigious role in one of the most deplorable companies in the modern world (at least from an ethical perspective). He also like attention and public display of status: he can’t resist to pick a fight with Gary. From a pure research perspective he’s long dead.
The question is: do we have enough of those two? Can we move on? Thanks.
Here's what I (a person with a fairly superficial understanding of how AI works, for context) can't understand - why wouldn't it have been trivial to prevent the kinds of issues that arose by just training the thing to only pull information from scientific papers/literature, and when a question arose on a topic that didn't have information in those places just say "we can't find enough information about that topic"?
It seems like it was good at structuring the writing, both at the article and sentence levels, and for valid prompts it produced accurate responses. But if you entered "write a wiki article about the Alien vs. Predator hypothesis," it would structure it like a wiki article about a scientific topic but just put random AVP stuff it found on the internet into that structure. Why couldn't they just explicitly define which sources are appropriate to pull information from? That seems easier to me than building the actual product they made (but again, I am a layman here).
[+] [-] dang|3 years ago|reply
Why Meta’s latest large language model survived only three days online - https://news.ycombinator.com/item?id=33670124 - Nov 2022 (119 comments)
[+] [-] dekhn|3 years ago|reply
It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
[+] [-] kspacewalk2|3 years ago|reply
Very similar to crypto evangelists boldly proclaiming the world of finance as obsolete. Rumours of you understanding how the financial system works were greatly exaggerated, my dudes.
[+] [-] nn3|3 years ago|reply
He's supposed to agree with you, or not express an opinion? Anything else short of this would be "defensive" right?
This whole idea that defending your positions in arguments is somehow a bad thing is a really odd modern development that I never understood.
[+] [-] short_sells_poo|3 years ago|reply
For someone who dedicated their career to ML, they'll naturally try to solve everything in that framework. I observe this in every discipline that falls prey to it's own success. If there's a problem, those in the industry will naturally try to solve it with ML, often completely ignoring practical considerations.
Is the engine in your car underperforming? Let's apply ML. Has your kid bruised their knee while skating? Apply ML to his skating patterns.
The one saving grace of ML is that there are genuinely useful applications among the morass.
[+] [-] vishnugupta|3 years ago|reply
Him and Grady Booch recently had a back and forth on the same subject on Twitter where to me it seemed like he couldn’t answer Booch’s very basic questions. It’s interesting to see another person with a similar opinion.
[+] [-] wpietri|3 years ago|reply
For sure. If this is how he treats outside experts, I can't imagine what it's like to work for him. Or rather, I can imagine it, and I think it does a lot to explain the release-and-panicked-rollback pattern.
[+] [-] warinukraine|3 years ago|reply
[+] [-] bjelkeman-again|3 years ago|reply
[+] [-] LegitShady|3 years ago|reply
You interrupting the messaging ruins it, so you get some deniable boilerplate response. its not personal.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] uJustsaidit|3 years ago|reply
Society is still led by vague mental imagery and promises of forever human prosperity. The engineering is right but no one asks if rockets to Mars are the right output to preserve consciousness. We literally just picked it up because the elders started there and later came to control where the capital goes.
We’re shorting so many other viable experiments to empower same old retailers and rockets to nowhere.
[+] [-] Simon_O_Rourke|3 years ago|reply
Like many thin-skinned hype merchants with a seven-figure salary to protect, they're going to try and block criticism in case it hits them in the pocket. Simple skin in the game reflex that will only hurt any chances of improvement.
[+] [-] geertj|3 years ago|reply
[+] [-] theptip|3 years ago|reply
Being constantly sniped at probably puts you in a default-unreceptive state, which makes you unable to take on valid feedback, as yours sounds like.
[+] [-] musicale|3 years ago|reply
Ideally scientists would be interested in the truth and engineers would be interested in making the system better.
[+] [-] fmajid|3 years ago|reply
[+] [-] thenightcrawler|3 years ago|reply
[+] [-] regnull|3 years ago|reply
[+] [-] gtmitchell|3 years ago|reply
And my expectations were exceeded by the first speaker. I couldn't wait for 3 full days of this! Then the second speaker got up, and spent his entire presentation telling why the first person was an idiot and totally wrong and his research was garbage because his was better. That's how I found out my field of study was broken into two warring factions, who spent the rest of the conference arguing with each other.
I left the conference somewhat disillusioned, having learned the important life lesson that just because you're a scientist doesn't mean you aren't also a human, with all the lovely human characteristics that entails. And compared to this fellow, the amount of money and fame at stake in my tiny field was miniscule. I can only imagine the kinds of egos you see at play among the scientists in this article.
[+] [-] PaulHoule|3 years ago|reply
But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
Let's hope "longtermism" and other A.I. hype goes the way of "Web3".
[+] [-] lupire|3 years ago|reply
[+] [-] thwayunion|3 years ago|reply
Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want.
Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want.
Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want.
Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want.
Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want.
Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.
[+] [-] naasking|3 years ago|reply
It boggles my mind how anyone can think otherwise. Existential dangers of superintelligent or even non-intelligent AI are the long-term result of the dangers of AI being developed and misused over time for human ends.
It's the exact same argument behind why we should be trying to track asteroids, or why we should be trying to tackle climate change: the worst-case scenario is unlikely or in the future, but the path we're on has numerous dangers where suffering and loss of human life is virtually certain unless something is done.
[+] [-] bondarchuk|3 years ago|reply
I hesitate to say "safe space", but... what if a group of people wants to come together discuss AI safety? If they'd have to regurgitate all the arguments and assumptions for everyone who comes along they'd never get anything done. If you are really interested to know where they are coming from, you can read the introductory materials that already exist. If the 99.9% of the world is hostile towards discussing AI safety (of the superintelligence explosion kind, not the corporate moralitywashing kind) there is some value in a place which is hostile to not discussing it, so that at least those interested can actually discuss it.
[+] [-] c1ccccc1|3 years ago|reply
Is that actually true, though? It's true that a higher fraction of the people in that community give credence to the intelligence explosion hypothesis than pretty much anywhere else. (This is what one would expect, since part of the purpose of LessWrong is to be a forum for discussions about super-intelligent AI.) But even if the intelligence explosion is a terrible, absolutely-wrong theory, that doesn't prevent the people who hold it from being open-minded and tolerant of independent thought. Willingness to consider new and different ideas is something the LessWrong community claims to value, so it would be a little bit weird if they were doing way worse than average at it. And AFAICT, it seems like they're doing fine. Some examples:
- Here [1] is a post critical of the intelligence explosion theory. It has 81 upvotes as of this writing, and the highest upvoted comment goes like: "thanks for writing this post, it makes a lot of good arguments. I agree with these things you wrote" (list of things) "here are some points where I disagree" (list of things). This may even be the original post you were talking about in your comment, except that it doesn't start with an apology.
- LW has 2 different kinds of voting: "Regular upvotes" provide an indication of the quality of a post or comment, and "agree/disagree votes" let people express how much they agree or disagree with a particular comment. Down-voting a high quality comment just because you disagree (instead of giving it a disagree-vote) would be against the culture on LW.
If you're already sure that LW is wrong about superintelligence, and you're trying to explain how they became wrong, then "those LW people were too open minded and fell for that intelligence explosion BS" makes more sense to me than anything about suppression of independent thought.
[1] https://www.lesswrong.com/posts/zB3ukZJqt3pQDw9jz/ai-will-ch...
[+] [-] creatonez|3 years ago|reply
Well that's an interesting way to misrepresent an entire important field of research based on what a few idiots said. There are serious people in that field who aren't addicted to posting on LessWrong.
[+] [-] scifibestfi|3 years ago|reply
Suffice it to say, ideologues desperately want control over everything AI/ML. That's the real danger.
[+] [-] cultureulterior|3 years ago|reply
[+] [-] tylerneylon|3 years ago|reply
I read the MIT Technology Review article, and I was asking myself “what is an example of Galactica making a mistake?” The article could easily have quoted a specific prompt, but doesn’t. It says the model makes mistakes in terms of understanding what’s real/correct or not, but the only concrete example I see in the article is that the model will write about the history of bears in space with the implication that it’s making things up (and I believe the model does make such mistakes). I don’t think it’s a good article because it’s heavy on quoting people who don’t like the work and light on concrete details.
Does the imperfection of a language model really mean the model should not exist? This seems to be what some critics are aiming for.
[+] [-] megaman821|3 years ago|reply
[+] [-] alpaca128|3 years ago|reply
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".
[+] [-] kspacewalk2|3 years ago|reply
Absolutely they can, and his employer could have kept it up. The issue is the phantasmagorical and ridiculous claims about AI-generated scientific research that LeCun peddles. When there's something concrete one can use to test these extremely bold claims, there's a way to at least partially apply a reality check to the claims, and demonstrate their ridiculousness. Which is a very useful and important part of how the scientific field evolves and advances. Feeding non-experts all these wild claims in perpetual future tense only works for so long, and it ought to be that way.
[+] [-] photochemsyn|3 years ago|reply
[+] [-] motohagiography|3 years ago|reply
This whole incident was a case study for product management and startup school 101. I've made this exact same category of error in developing products, where I said, "hey, look at this thing I built that may mean you don't have to do what you do anymore!" and then was surprised when people picked it apart for "dumb" reasons that ignored the elegance of having automated some problem away.
If this model were really good, they would have used it to advance a bunch of new ideas in different disciplines before exposing it to the internet. Reality is, working at Meta/Facebook means they are too disconnected from the world they have influenced so heavily to be able to interpret real desire from people who live in it anymore. When you are making products to respond to data and no actual physical customer muse, you're pushing on a rope. I'd suggest the company has reached a stage of being post-product, where all that is left are "solutions," to the institutional customers who want some kind of leverage over their userbase, but no true source of human desire.
[+] [-] belval|3 years ago|reply
The article really fails to explain that LeCun and Marcus have been trading insults for the last few years, it's hardly LeCun snapping at some random person.
[+] [-] logicalmonster|3 years ago|reply
If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."
[+] [-] A4ET8a8uTh0|3 years ago|reply
Please correct me if I am misrepresenting a chain of events here.
A.. tool lands that allows one use language model to generate content. People feed it false data and share that, surprise, data it produces from the data the model is based on is false. How is this a surprise? I am still not sure why Meta would pull it? It can still be useful, but it was made not useful. I am not sure what a proper metaphor is for it, but it is almost like I give you a tool ( lets say a knife ) and you complain that the tool produces bad results when drinking soup.
What am I missing here?
<<or maybe it [Galactica] was removed because people like you [Marcus] abused the model and misrepresented it. Thanks for getting a useful and interesting public demo removed, this is why we can’t have nice things.
<<Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts.
[+] [-] adamsmith143|3 years ago|reply
Also the title of this post is deliberately inflammatory. Should be more like "Head of team that spent months building complex ML system annoyed when people spend undue amounts of time criticizing it."
[+] [-] ak_111|3 years ago|reply
Their abstract says "In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
If I was a reviewer of this paper I would ask them to add (if they haven't so) significant section to the body of the paper highlighting the limitation of the model and the ways it can be misused. Including showing examples of wrong output.
I would then ask them to rewrite the abstract to include something along the lines "We also highlight the limitations of the model including inability to distinguish fact from fiction in several instances and the ways it can be misused and outline some ideas on how these limitation could be mitigated or overcome in the future."
[+] [-] chubot|3 years ago|reply
Isn't this the same problem that Github Copilot has?
Fundamentally it has no idea whether code works. It doesn't even know what the problem is.
It just spits out things that are similar to things its seen before, including buggy code from Github repositories.
Not sure why it's so popular. I guess it helps you write status quo code faster (the status quo being buggy and slow) -- I would rather it help us write better code.
[+] [-] seydor|3 years ago|reply
Lecun implied on twitter that they 'll get it back. I really hope so
[+] [-] Cyberdog|3 years ago|reply
Well, that matches our current experience with human-written Wikipedia articles pretty closely then.
[+] [-] fumeux_fume|3 years ago|reply
[+] [-] LLMscientist|3 years ago|reply
Not only did they exaggerate and hype, but they also didn't even try to solve some of the most glaring issues. The efforts on toxicity mentioned in their paper aren't even mid. They barely put effort into measuring the issue, and definitely didn't make any attempt to mitigate or correct.
Toxicity isn't really the point. Here's the point. If you can't prevent a model from being overtly toxic, then why should I believe you can give any guarantee at all about the model's output? I shouldn't, because you can't.
Galactica is just another a language model. It can be a useful tool. Facebook and LeCun oversold its capabilities and downplayed its issues. If they had just been honest and humble, things would've probably gone very differently.
In some sense, this is good news. The deep learning community -- and generative model work in particular -- is getting a much-needed helping of humble pie.
Hopefully we can continue publishing and hosting models without succumbing to moral panic. But the first step toward that goal is for scientists to be honest about the capabilities and limitations of their models.
----
My account is new so I am rate limited and unable to reply to replies. My response to the general vibes of replies is therefore added to the above post as an edit. Sorry.
Response about toxicitiy:
It's a proxy that they say they care about. I can stop there, but I'll also point out: it's not just "being nice", it's also stuff like overt defense of genocide, instructions for making bombs, etc. These are lines that no company wants their model to cross, and reasonably so. If you can't even protect Meta enough to keep the model online for more than a day or two, then why should I believe you can give any guarantee at all about the model's output in my use case? (And, again, they can't. It's a huge problem with LLLMs)
Response about taking the model down:
I'm not at FB/Meta, but I think I know what happened here.
In the best case, Meta was spending a lot of valuable zero-sum resources (top of the line GPUs) hosting the model. In the worst case they were setting a small fortune on fire at a cloud provider. Even at the largest companies with the most compute, there is internal competition and rationing for the types of GPUs you would need to host a Galactica-sized model. Especially in prototype phase.
An executive decided they would rather pull the plug on model hosting than spend zero-sum resources on a public relations snafu with no clear path to revenue. It was a business decision. The criticism of Galactica and especially the messaging around it was totally fair. The business decision was rational. Welcome to private sector R&D; it works a little different from your academic lab for better and for worse.
[+] [-] MikeTheGreat|3 years ago|reply
One of the things I really like about HN is the _lack_ of clickbait titles. Some titles are more informative, some less, but overall I feel like the titles are clear, to the point, and not carefully crafted/engineered to poke the lizard part of my brain in the way that clickbait titles are.
Disclaimer: I haven't read the article so I can't propose a title myself. And with a title like this I'm not going to.
[+] [-] seydor|3 years ago|reply
[+] [-] ChrisMarshallNY|3 years ago|reply
[+] [-] michele_f|3 years ago|reply
Yann Lecun, which I personally met a couple of times, is in a way another sort of typical character: the ever-childish researcher that likes money a lot, to the point of accepting a prestigious role in one of the most deplorable companies in the modern world (at least from an ethical perspective). He also like attention and public display of status: he can’t resist to pick a fight with Gary. From a pure research perspective he’s long dead.
The question is: do we have enough of those two? Can we move on? Thanks.
[+] [-] awillen|3 years ago|reply
It seems like it was good at structuring the writing, both at the article and sentence levels, and for valid prompts it produced accurate responses. But if you entered "write a wiki article about the Alien vs. Predator hypothesis," it would structure it like a wiki article about a scientific topic but just put random AVP stuff it found on the internet into that structure. Why couldn't they just explicitly define which sources are appropriate to pull information from? That seems easier to me than building the actual product they made (but again, I am a layman here).