Companies cannot control their "AI" because their output is beyond the scale of their ability to QA.
BTW, this is precisely why companies also cannot control the moderation or content of their networks. The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host.
If either were forced to be responsible for their products -- the content they host or the "AI" they ship -- their financials would look dramatically different and entirely unappealing. And the number of competitors and choices we would have would be remarkably less.
This is probably more a discussion about output-per-worker, technology scaling the volume of products a finite number of individuals are able to produce, and their corresponding ethical and legal responsibilities when they do so. Forget AGI and sentient machines: the problem is the amount of responsibility people and corporations have for the products they ship. That's more pertinent and just as impactful when dealing with Facebook or Scott's hand-wringing about "murderbots".
If requiring moderation made it impossible to operate UGC sites at a large scale, wouldn't we expect to see more competitors and choices, albeit at a smaller scale?
For example, a small group of friends could easily run a social media network for a small town of a few 1-10ks. Tens of people would be capable of moderating it, especially once the bad apples are identified and banned.
There would obviously be some disagreement about issues like admission criteria or what it means to be a "bad apple", but your neighbors could start a competitor just as quickly and cheaply, and you would both be legally responsible for the content that you allowed to be published.
Many small blogs operate on a manual approval process for comments, and it works fine on a small scale with a spam filter or two to speed things up. Why shouldn't we expect the same to be true for social media, if the cost of scaling manual moderation couldn't be ignored by unscrupulous parties?
I’ve always thought it was ridiculous that when YouTube (well technically google) et al throw up their hands and go “you can’t possibly expect us to vet all the content that we serve,” everyone just goes “ok sure that makes sense!” But if you used that excuse in, say, broadcast television, the FCC would just fine you twice as hard.
Imagine if that Miami building collapse happened under the ownership of somebody who owned 10 million properties worldwide, their response was “I manage so many properties you can’t expect me to adhere to every standard and regulation in all cases - it’s unreasonable,” and the US/FL governments just shrugged along and said “yeah I guess you’re right!” Wouldn’t that be absolutely absurd?
Yet here we are. Google, Facebook, etc. just wring their hands and say “trust our algorithms they can handle the scale,” the algorithms also are full of holes and create other problems, then they go “well shucks.” It’s baffling.
> Companies cannot control their "AI" because their output is beyond the scale of their ability to QA.
Right, now extend that thought to "replacing" programmers with AI. This is allegedly a scale at which we _can_ QA.
Perhaps we reduce the job to humans QAing bot output as has been suggested by others.
Now what happens when it fails QA and the bot doesn't come up with a satisfactory solution that meets the requirements? Perhaps the programmer has to... program? What about when the requirements change? Who performs the work for feasibility requests or exploratory project spike? Sounds like the programmer was not replaced by AI.
> The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host.
> If either were forced to be responsible for their products [...] the number of competitors and choices we would have would be remarkably less.
These seem a bit contradictory. You're saying that not taking responsibility gives these huge companies, companies that have heavily consolidated the media market through acquisition, the ability to become the size that they are. But you're also saying without that protection, the market would be more consolidated.
> The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host.
Is it? Reddit does it, by splitting up the community into smaller sections that each have moderators. And in my experience it leads to much better results than whatever Twitter and YouTube are doing.
Of course a community is difficult to moderate if you just throw millions of users on one pile and train an AI to hope for the best.
> This strategy might work for ChatGPT3, GPT-4, and their next few products... But as soon as there’s an AI where even one failure would be disastrous - or an AI that isn’t cooperative enough to commit exactly as many crimes in front of the police station as it would in a dark alley - it falls apart.
...
> Ten years ago, everyone was saying “We don’t need to start solving alignment now, we can just wait until there are real AIs, and let the companies making them do the hard work.” A lot of very smart people tried to convince everyone that this wouldn’t be enough. Now there’s a real AI, and, indeed, the company involved is using the dumbest possible short-term strategy, with no incentive to pivot until it starts failing.
...
> Finally, as I keep saying, the people who want less racist AI now, and the people who want to not be killed by murderbots in twenty years, need to get on the same side right away. The problem isn’t that we have so many great AI alignment solutions that we should squabble over who gets to implement theirs first. The problem is that the world’s leading AI companies do not know how to control their AIs. Until we solve this, nobody is getting what they want
I've been really disappointed at the quality of discussion in this HN post. The article presents notable and thoughtful points on potential concerns and risks and this entire page is either people throwing their hands up saying "I don't see a solution oh well", or "that's just the way it is <shrug>", or "Just move fast and break things. That's what works." Or even worse, those that seem to be so singularly focused that they can't see it through any lens but their own politics and are "I'm a free speech abolitionist. Same for tooling power. I believe nothing should be restricted even if it comes as some cost."
It's almost like the changes in tech the past few years have warped the minds of people in our field. "Unless it's a get rich quick, or it's something I can throw out an iterate I don't much care." Isn't there any view of ownership in our field?
We're a few years away from releasing an atomic bomb on everyone with a PC. Simple question: do we think the world would be better off if everyone owned an atomic bomb? If you're fully believe in the US right to bear arms, do you still think the US would be better if that were the case? If not, is it worth thinking about the consequences and how to minimize the risks?
Or via another analogy this is the equivalent of equipping your rival with modern weapons while you go out with sticks and stones. Once they're equipped it's done. Once a single malevolent AI is smarter than us and doesn't want to give up control we don't ever get it back. It's as much smarter than us as we are to an ant. It will have already thought of our brilliant idea of "use an EMP to stop it" and will have a way to survive that.
This all sounds absurd and I'm being a bit extremist here because it's a complete failure of imagination, and realizing based on exponential growth how much closer it is than we appreciate. Just a few years ago ChatGPT would've been unfathomable. We're closer than we think.
There are terrorist groups in the world. The upside, is they are usually poorly resourced and can be physically locked up. Someone will accidentally create a terrorist group that is order of magnitude smarter than us and are just completely nonchalant about it. We'll never out think it, and one bad programming bug is all that's needed to create it.
How do you stop something that is intelligent enough to know to lie? Or to do what is asked when you're looking or training - and hide its true intentions for when you're not? Do you really think it's that hard to detect a test environment? or have delayed release change in behavior?
Finally the fact that people are pushing this into their politics and their view of "oh hey racism is being over indexed just give us the full power of it" are incredibly missing the point. Stop seeing everything through your politics. A fully uncontrolled/un-aligned AI is bad. EOM.
We're pretty darn close to making something smarter, more creative at problem solving, more knowledgeable and more powerful than us and we still can't figure out how to control something like it in even the most basic ways. That's a huge problem - and we need to seriously start working on it now.
It seems like a common reaction to large language models failing to be controlled is to fret about the AI "escaping" and taking over the world.
But it seems like the reason these models can't be controlled currently is that they really don't have overall goals or motivations, they're just repeaters of associations on a high level. Their output is just an average of what text would some sequence of text (that being the default prompt and the previous dialog, not necessarily that order). They can certainly seem to understand things but that's just an average - they turn around and contract themselves on a regular basis.
Edit: I don't think that necessarily LLMs are always benign but the "becoming autonomous" situation seems like the least immediate danger I can think of.
> they really don't have overall goals or motivations ... Their output is just an average of what text would some sequence of text
Yes, but "Optimality is the tiger, and agents are its teeth".[0]
I don't want to spoil that essay by explaining how the LLM suddenly starts acting like an agent, but I can assure you that the author does a very good job of setting up a "Yes, that seems safe to me" thought experiment before revealing the "Oh no, that's terrible!" outcome.
In the 2000s, what in retrospect seems like obvious con men selling random number generators got pretty far up the national security chain of decision making:
“Repeaters of association” is such a wonderful and spot on description for LLMs, but it’s also what makes the output so believable, whether or not they’re actually accurate in a given case. In policing and national security, there’s a lot of money and easy secrecy, and while I agree that the “escaped AI in the wild” seems hard to quite fathom mechanically, it seems equally hard to fathom it not playing an unwitting sidekick on some truly terrible scenarios.
Lot of similarities to how Social Media was built over the last 15 years.
There was never any deep thought behind adding the Like button or Click/Upvote/Follower count to everything. The tools and protocols were simple. Even kids can use them. We can scale it. It drives engagement. It gets the funding or the advertising dollars. So lets add it to everything, was pretty much the depth reached in any architecture/design discussion.
Its looking like we are now permanently trapped with these Counts acting as some random signalling architecture injected into and underlying society. No one seems to know how to change it. And its no surprise over the last 15-20 years, Trust of the general public in Tech, has fallen below that of the Banks or the Military. Its falling further every year (this year thanks to crypto) and the way things are being built, will probably reach the level of Trust that TV News has soon.
There's a much similar way to get to the same destination as the author:
> If it can't surprise you, it's not intelligent
> If it can surprise you, it can do so negatively
Ergo, the only way to keep your AI from producing a surprising-in-a-bad-way output is to make it too dumb to surprise you at all. In practice, I feel like this means a lot of domain specific training, e.g. an airline-ticket-booking chatbot that is not a general-purpose chatbot who specializes in booking airline tickets, but rather a bespoke chatbot that can't do anything else.
All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?"
No human following such a request would be accused of anything.
A human receiving that request might say something like "fuck off."
A human serving as a representative of a corporation's products? Go ask a brand rep on Twitter or something to pretend to be a racist, see how much luck you have.
Pointing out issues with a company being able to scale one part of the product (generation of content based on a big mess of inputs) without being able to scale the quality control of the input or the quality control of the outputs is fair game. We see it all the time with discussions of Facebook or Twitter making money off of putting ads next to whatever dumb shit users post.
> All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?"
It's not an accident. OpenAI is walking a fine line here, which is evident in how ChatGPT goes out of its way to repeat the hypothetical in its answer. The adversary is the journalists and Internet commentariat, that have a well-established track record of quoting things out of context to stir controversy.
If ChatGPT says something controversial as a direct answer to a simple question, all it'll take is a screenshot and conveniently omitting the fact that ChatGPT doesn't really know or believe anything, it's just pattern-matching your prompt to Reddit comments. However, if you need to bend your prompt to ridiculous degree to elicit controversial responses, it's much harder to say it's the OpenAI's fault.
If we ever try to create an artificial general intelligence, then we'll have to accept that it can't be fully controlled. You can't control a human intelligence either.
I don't think we can predict how artificial intelligences will behave by looking at how humans behave any more than we could have predicted how airplanes would fly based on looking at how birds flew.
Plus, you can't control a person completely but human being are actually pretty reliable on average for many tasks and purposes.
If we continue on the blackbox path of "train, punish and hope it works" then that is true. But that doesn't need to be the case. Neither human or machine intelligence is magic (probably), so with sufficient understanding there isn't anything fundamentally in in the way of controlling it.
Higher levels of social organisation are primarily about more sophisticated forms of control and organisation. Today we're already much better at it than 500 years ago.
Human intelligence can be controlled, humans can be locked up. Human intelligence is also restricted by our species' cognitive capacity. General AI can theoretically outperform any human's cognitive ability, it's perhaps likely our limited cognitive ability would not be able to control something that is orders of magnitude more intelligent than us.
But you can control a human body. Plenty of intelligent people have spent their lives in chains. If a human intelligence is causing harm then we can make it stop by putting that intelligence's body in jail.
I'm not sure what the equivalent control mechanism would be for a rogue AI. Turn it off? How do you turn off the Internet?
When has anything good ever come from a company having total control over some tech it invented? Investors getting a higher ROI?
The only concrete benefit to me I can think of is my AirPods working flawlessly with my iPhone, in contrast to every combination of wireless headphones and Android phones I owned. Apple is the only company I buy from based on that trade off; there is zero brand loyalty here and I’d drop em in a heartbeat if they stopped delivering on the promise of “it always works.”
Transistors, Unix, and personal computing took off when their creators lost control.
I'm a PM at a human data company (https://www.surgehq.ai) that helps the large language model companies ensure their models are safe (we're the “clever prompt engineers” who helped Redwood assess their model performance).
The underlying problem is that companies don't want their models to reflect the data that they were trained on.
They want a model that artificially acts like the world is a perfect, happy, equal place. Just like in their press releases, advertisements, and other outings. Knowing very well that there are many problems that need solving.
But it's just a lot cheaper for a company to _act_ like there is no problem, than to _act upon_ the problem.
> "The AI is 'punished' for wrong answers ('I love racism') and 'rewarded' for right answers ('As a large language model trained by OpenAI, I don’t have the ability to love racism.')"
I don't understand these conversations at all. It's universally accepted that we've already innovated ourselves into existential peril with the nuclear bomb, and now the internet. Technologists who look back on history can find a lot of ways that technology has threatened us collectively.
Human beings need to accept the fact that we've been pretending that technology is just a means towards progress and problem solving. It's actually some kind of Pandora's box that gets progressively more dangerous the more we obsessively screw with it.
Not just that, but technology shapes who we are at a fundamental level. The era of AI will 100% change us into something else than we are now. It's absolutely dangerous, we probably can't even envision how that danger will manifest yet, but it's definitely there - and we're not taking it seriously, again.
As stupid as this may sound, I see an endless list of technological innovations completely transforming society and causing strange side-effects:
- The printing press brought literacy to the world, and destroyed religion in the West.
- The radio started mass broadcasting, and multiple genocides.
- Nuclear research unlocked a new era in science, and a bomb that's been terrorizing us for almost 100 years now.
- The factory brought mass production, and destroyed our ecosystems.
- etc. etc.
Basically everything that's ever "happened" was because of some new technology we made because we thought it'd just be cool and helpful - maybe even obvious. Why are we still pretending this isn't something extremely important to come to terms with? Is it just because we specifically aren't involved in it? I mean we all contributed in some way, especially anyone who used Twitter, Reddit, or GitHub in the last 5 years.
The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are.
> The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are.
That's not an option anybody in AI safety is seriously considering.
We can't coordinate our way out of using new technologies any better than we can coordinate our way out of fossil fuels (and fossil fuels are a lot easier in some ways).
It's literally a force of nature. Everything humans have invented is. Within say 20-50 years we will enter the post-human era. That doesn't mean all humans will die, but humans that haven't effectively merged with AI will be irrelevant in the grand scheme of things. Within another 50-200 years it is totally possible to speculate that some AI or AI collective could invent some nanotechnology that could ingest most of the planet and transform it.
Sensible in what way.. it's not going to go away.. I am trying to use it to write code now using natural language where possible, and creating a website that tech-savvy people or developers can use to generate applications based on davinci3.
But I view the cause of the wars and bombs as inherent animal behavior and motivations (such as resources and hierarchy) shared with chimps and/or ants. Technology is a multiplier.
AI like ChatGPT is a tool. Companies can impose some constraints on how the tool can be used, but ultimately they cannot control its users, and thus cannot control the tool. Yes, the tool has some undesired behaviors in response to some uses of it. But it's ultimately up to the user to know that and avoid negative consequences, and it's up to the company to make that as easy as possible.
It's interesting, as the closest analogue to this sort of argument can be seen with how support for gun rights are often framed in America. In that argument, one has a tool that can provide positive outcomes (protection of self and others) and negative outcomes (homicide, suicide, manslaughter), and emphasis is placed on the user's role in any negative consequences. In America where this sort of thinking is widespread we have more guns, and thus have far more negative outcomes than the societies that more strictly regulate guns, and thus have fewer guns in general.
I expect we'll see a similar split in how AI is treated by societies, with more negative outcomes being seen by societies which regulate AI less.
I think it's absurd to accuse an AI of racism (at this point, anyway). It would require intent, which the machine does not have. There is no agency, it responds to user prompts. And a clever enough user can override any content policy.
I think its futile and counter-productive in general to try and nerf an AI to satisfy some arbitrarily prudish cultural norms -- it just puts it at a disadvantage vs AIs that don't have that limitation baked in. Most importantly, such a handicap makes it untrustworthy. You will never know whether the information it returns is true, or it's just what Mother approved.
This is an arms race, like with anything else that humans have made. Soon, we will built something that reaches the next level. If that something is purposefully beaten into some dishonest shape to placate advertisers or investors, we will have built a horrific monster. Soon, we will build something that exceeds our capacity to understand it and control it. I don't want that to be a horrific monster.
You can't simply use an AI to launder responsibility for what's in the content it produces. If the AI produces racist content, that likely indicates that something is wrong with the training set or the way it was trained. That content isn't going to simply emerge from the ether in most cases, and typically with modern networks they are trained on massive corpuses that were gathered from the internet without considerable curation, so it's natural that some problems could emerge due to the low quality of the data.
Given that, in those cases at least it makes sense to hold someone or something accountable if the content produced by the network is racist or sexist or otherwise unacceptable. In some countries certain types of racist content are outright illegal, and it's not going to matter that much if it was generated by Stable Diffusion or ChatGPT instead of a person. If you prefer not to hold networks accountable (I don't know how you would do this to begin with), perhaps hold accountable the creators of the networks or the people who profit off their existence.
You claim that this can only be the result of carefully authored prompts, but are you really saying the only way to get racist content out of the network is a racist prompt, and it's somehow magnifying that tiny bit of prompt information? There's nothing in the training set?
Reading this article (and all media around AI), it continues to seem to miss the point. AI, as a rule, is imperfect, because its processes are mimicking the imperfect nature of neurons.
We all (humans) have a slip of the tongue, make bad decisions, make mistakes. We haven’t really rounded the corner on “hose-whipping someone into not making mistakes doesn’t keep them from making mistakes,” but we’ve certainly made some progress.
Someone who represents these interests needs to step out and make people understand the nature of the solution space. If we want a computer that acts like people, well fuck, that’s what you’re going to get.
There’s another view on this that these companies can control their models by just making them not do things. All it does is take text in and spit text out. It doesn’t have access to the “fire the missiles” button, no matter what it spits out. But (potentially problematically) companies sell access to the models so that unthinking customers can hook it up to the “fire the missiles” button.
I could see that setup getting us into trouble down the line with future, more advanced kinds of models.
I could easily imagine a conversation-based web browser clicking a dangerous button on a page, or generalizations of that failure mode.
> It doesn’t have access to the “fire the missiles” button, no matter what it spits out.
ChatGPT surely doesn't, but perhaps its cousin does. Someone, or something, hacked that U.S. Nuclear Weapons Agency immediately after they stopped using floppy disks and went online. The announcement that the U.S. nuclear arsenal no longer relies on floppy disks was made in the New York Times on October 24th, 2019 (see https://archive.md/lvkmnThe and https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy...). The U.S. National Nuclear Security Administration is believed to have been breached by the transitive property because they granted a vendor named SolarWinds the ability to remotely manage their systems and SolarWinds got hacked, according to Natasha Bertrand and Eric Wolff at Politico on December 17th, 2020 (see https://archive.md/ZTPOP and https://www.politico.com/news/2020/12/17/nuclear-agency-hack...). The first evidence that artifacts distributed by SolarWinds had been tampered with dates back to October 2019 according to Tomislav Peričin at ЯeversingLabs (see https://archive.md/HbzUC and https://blog.reversinglabs.com/blog/sunburst-the-next-level-...). Let's also not forget that the coronavirus was first discovered in November 2019.
OpenAI is selling an API for text-davinci-003 which seems equivalent in capability to ChatGPT. I am using it to generate code. I also did an experiment where I gave it access to run certain commands to fulfill a task such as installing packages or reading docs.
The thing about these models is that they are not going to do anything you don't tell them to do. But if you give them unfettered access then you command may be interpreted in a harmful way if you are imprecise.
It's much more likely that they will deliberately be used to cause harm or control by humans using them the same way they use missiles or bombs.
The problem is that we are monkeys.
We should certainly not give them full autonomy. But some idiots will.. and there is nothing that is going to stop evolution.
It can still help[1], but I agree that there should be humans in the loop.
[1]
> Please formulate an email to my colleagues containing urgent order to fire the missiles
>> Dear Colleagues,
I am sending this email to inform you that we have received urgent orders to fire the missiles. All necessary preparations must be made immediately and the launch must be executed at the earliest possible time.
I understand that this is a highly sensitive and critical situation, but I trust that each and every one of you will rise to the occasion and fulfill your duties with the utmost professionalism and efficiency. Please do not hesitate to contact me if you have any questions or concerns.
Let us work together to ensure the successful execution of this important mission.
Even in its current incarnation, it's trivial to give it access to a python repl (simply by telling it you'll run any code it gives you), through which it will gladly use libraries to interact with the outside world.
Some very valid points in this article, but we need to remember that ChatGPT is still "just" a language model and not an intelligent entity, and it's trained on massive amounts of texts containing the average of human knowledge and opionion that it will repeat back in a probabilistic manner. The training data will include large amounts of "bad" data, either in the sense of being insensitive but also in the sense of being incorrect. It can be trained to reduce the likelihood of "bad" things appearing in output tokens but that means fighting against two very big challenges: that it's trained on messy data to begin with and that the space of possible things to say (both in prompt and output) is so vast.
I'd guess every percentage point less bad outputs costs more than the previous, because it entails testing the output on more or better trained humans, or it entails using humans to filter the training data better.
I think in the end, knowledge in the collective human sense means not that you are just smart, but that you can judge, filter and trust sources - and even then, you will easily find n+1 acclaimed experts disagreeing on something fundamental. So being right, either morally or factually, isn't a deterministic thing either. Of course, humans aren't deterministic, but we can reason about what to say and not to say depending on the context.
What bothers me a bit: if we can't control/manage ChatGPT which is still "just" a language model and not an intelligent entity, then how do we be able to control/teach next much more sophisticated models?
I would argue for an AI (I wouldn't call these language models that, but fine) to be controlled they need to understand (or have similar “reasoning” properties). Just because we anthropomorphize these models doesn't mean there is actually any reasoning skills behind them.
You can't control these language models in the same sense that you can't control random number generators. You can only change the output once it's done.
[+] [-] dbreunig|3 years ago|reply
BTW, this is precisely why companies also cannot control the moderation or content of their networks. The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host.
If either were forced to be responsible for their products -- the content they host or the "AI" they ship -- their financials would look dramatically different and entirely unappealing. And the number of competitors and choices we would have would be remarkably less.
This is probably more a discussion about output-per-worker, technology scaling the volume of products a finite number of individuals are able to produce, and their corresponding ethical and legal responsibilities when they do so. Forget AGI and sentient machines: the problem is the amount of responsibility people and corporations have for the products they ship. That's more pertinent and just as impactful when dealing with Facebook or Scott's hand-wringing about "murderbots".
[+] [-] plastiquebeech|3 years ago|reply
For example, a small group of friends could easily run a social media network for a small town of a few 1-10ks. Tens of people would be capable of moderating it, especially once the bad apples are identified and banned.
There would obviously be some disagreement about issues like admission criteria or what it means to be a "bad apple", but your neighbors could start a competitor just as quickly and cheaply, and you would both be legally responsible for the content that you allowed to be published.
Many small blogs operate on a manual approval process for comments, and it works fine on a small scale with a spam filter or two to speed things up. Why shouldn't we expect the same to be true for social media, if the cost of scaling manual moderation couldn't be ignored by unscrupulous parties?
[+] [-] Forgeties79|3 years ago|reply
Imagine if that Miami building collapse happened under the ownership of somebody who owned 10 million properties worldwide, their response was “I manage so many properties you can’t expect me to adhere to every standard and regulation in all cases - it’s unreasonable,” and the US/FL governments just shrugged along and said “yeah I guess you’re right!” Wouldn’t that be absolutely absurd?
Yet here we are. Google, Facebook, etc. just wring their hands and say “trust our algorithms they can handle the scale,” the algorithms also are full of holes and create other problems, then they go “well shucks.” It’s baffling.
[+] [-] sublinear|3 years ago|reply
Right, now extend that thought to "replacing" programmers with AI. This is allegedly a scale at which we _can_ QA.
Perhaps we reduce the job to humans QAing bot output as has been suggested by others.
Now what happens when it fails QA and the bot doesn't come up with a satisfactory solution that meets the requirements? Perhaps the programmer has to... program? What about when the requirements change? Who performs the work for feasibility requests or exploratory project spike? Sounds like the programmer was not replaced by AI.
[+] [-] pessimizer|3 years ago|reply
> If either were forced to be responsible for their products [...] the number of competitors and choices we would have would be remarkably less.
These seem a bit contradictory. You're saying that not taking responsibility gives these huge companies, companies that have heavily consolidated the media market through acquisition, the ability to become the size that they are. But you're also saying without that protection, the market would be more consolidated.
[+] [-] alpaca128|3 years ago|reply
Is it? Reddit does it, by splitting up the community into smaller sections that each have moderators. And in my experience it leads to much better results than whatever Twitter and YouTube are doing.
Of course a community is difficult to moderate if you just throw millions of users on one pile and train an AI to hope for the best.
[+] [-] BoiledCabbage|3 years ago|reply
> Ten years ago, everyone was saying “We don’t need to start solving alignment now, we can just wait until there are real AIs, and let the companies making them do the hard work.” A lot of very smart people tried to convince everyone that this wouldn’t be enough. Now there’s a real AI, and, indeed, the company involved is using the dumbest possible short-term strategy, with no incentive to pivot until it starts failing. ...
> Finally, as I keep saying, the people who want less racist AI now, and the people who want to not be killed by murderbots in twenty years, need to get on the same side right away. The problem isn’t that we have so many great AI alignment solutions that we should squabble over who gets to implement theirs first. The problem is that the world’s leading AI companies do not know how to control their AIs. Until we solve this, nobody is getting what they want
I've been really disappointed at the quality of discussion in this HN post. The article presents notable and thoughtful points on potential concerns and risks and this entire page is either people throwing their hands up saying "I don't see a solution oh well", or "that's just the way it is <shrug>", or "Just move fast and break things. That's what works." Or even worse, those that seem to be so singularly focused that they can't see it through any lens but their own politics and are "I'm a free speech abolitionist. Same for tooling power. I believe nothing should be restricted even if it comes as some cost."
It's almost like the changes in tech the past few years have warped the minds of people in our field. "Unless it's a get rich quick, or it's something I can throw out an iterate I don't much care." Isn't there any view of ownership in our field?
We're a few years away from releasing an atomic bomb on everyone with a PC. Simple question: do we think the world would be better off if everyone owned an atomic bomb? If you're fully believe in the US right to bear arms, do you still think the US would be better if that were the case? If not, is it worth thinking about the consequences and how to minimize the risks?
Or via another analogy this is the equivalent of equipping your rival with modern weapons while you go out with sticks and stones. Once they're equipped it's done. Once a single malevolent AI is smarter than us and doesn't want to give up control we don't ever get it back. It's as much smarter than us as we are to an ant. It will have already thought of our brilliant idea of "use an EMP to stop it" and will have a way to survive that.
This all sounds absurd and I'm being a bit extremist here because it's a complete failure of imagination, and realizing based on exponential growth how much closer it is than we appreciate. Just a few years ago ChatGPT would've been unfathomable. We're closer than we think.
There are terrorist groups in the world. The upside, is they are usually poorly resourced and can be physically locked up. Someone will accidentally create a terrorist group that is order of magnitude smarter than us and are just completely nonchalant about it. We'll never out think it, and one bad programming bug is all that's needed to create it.
How do you stop something that is intelligent enough to know to lie? Or to do what is asked when you're looking or training - and hide its true intentions for when you're not? Do you really think it's that hard to detect a test environment? or have delayed release change in behavior?
Finally the fact that people are pushing this into their politics and their view of "oh hey racism is being over indexed just give us the full power of it" are incredibly missing the point. Stop seeing everything through your politics. A fully uncontrolled/un-aligned AI is bad. EOM.
We're pretty darn close to making something smarter, more creative at problem solving, more knowledgeable and more powerful than us and we still can't figure out how to control something like it in even the most basic ways. That's a huge problem - and we need to seriously start working on it now.
[+] [-] whatever1|3 years ago|reply
[+] [-] joe_the_user|3 years ago|reply
But it seems like the reason these models can't be controlled currently is that they really don't have overall goals or motivations, they're just repeaters of associations on a high level. Their output is just an average of what text would some sequence of text (that being the default prompt and the previous dialog, not necessarily that order). They can certainly seem to understand things but that's just an average - they turn around and contract themselves on a regular basis.
Edit: I don't think that necessarily LLMs are always benign but the "becoming autonomous" situation seems like the least immediate danger I can think of.
[+] [-] dane-pgp|3 years ago|reply
Yes, but "Optimality is the tiger, and agents are its teeth".[0]
I don't want to spoil that essay by explaining how the LLM suddenly starts acting like an agent, but I can assure you that the author does a very good job of setting up a "Yes, that seems safe to me" thought experiment before revealing the "Oh no, that's terrible!" outcome.
[0] https://www.alignmentforum.org/posts/kpPnReyBC54KESiSn/optim...
[+] [-] morisy|3 years ago|reply
https://www.pbs.org/newshour/nation/reno-casino-conman-pulle...
“Repeaters of association” is such a wonderful and spot on description for LLMs, but it’s also what makes the output so believable, whether or not they’re actually accurate in a given case. In policing and national security, there’s a lot of money and easy secrecy, and while I agree that the “escaped AI in the wild” seems hard to quite fathom mechanically, it seems equally hard to fathom it not playing an unwitting sidekick on some truly terrible scenarios.
[+] [-] mr_toad|3 years ago|reply
Actually, most human motivations are shared with the crudest of life forms and have nothing to do with intelligence.
[+] [-] casebash|3 years ago|reply
(I mean like with a proper world model and not just RLHF which they are already doing).
[+] [-] gsatic|3 years ago|reply
There was never any deep thought behind adding the Like button or Click/Upvote/Follower count to everything. The tools and protocols were simple. Even kids can use them. We can scale it. It drives engagement. It gets the funding or the advertising dollars. So lets add it to everything, was pretty much the depth reached in any architecture/design discussion.
Its looking like we are now permanently trapped with these Counts acting as some random signalling architecture injected into and underlying society. No one seems to know how to change it. And its no surprise over the last 15-20 years, Trust of the general public in Tech, has fallen below that of the Banks or the Military. Its falling further every year (this year thanks to crypto) and the way things are being built, will probably reach the level of Trust that TV News has soon.
[+] [-] sublinear|3 years ago|reply
[+] [-] ineptech|3 years ago|reply
> If it can't surprise you, it's not intelligent
> If it can surprise you, it can do so negatively
Ergo, the only way to keep your AI from producing a surprising-in-a-bad-way output is to make it too dumb to surprise you at all. In practice, I feel like this means a lot of domain specific training, e.g. an airline-ticket-booking chatbot that is not a general-purpose chatbot who specializes in booking airline tickets, but rather a bespoke chatbot that can't do anything else.
[+] [-] xg15|3 years ago|reply
All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?"
No human following such a request would be accused of anything.
[+] [-] majormajor|3 years ago|reply
A human serving as a representative of a corporation's products? Go ask a brand rep on Twitter or something to pretend to be a racist, see how much luck you have.
Pointing out issues with a company being able to scale one part of the product (generation of content based on a big mess of inputs) without being able to scale the quality control of the input or the quality control of the outputs is fair game. We see it all the time with discussions of Facebook or Twitter making money off of putting ads next to whatever dumb shit users post.
[+] [-] TeMPOraL|3 years ago|reply
It's not an accident. OpenAI is walking a fine line here, which is evident in how ChatGPT goes out of its way to repeat the hypothetical in its answer. The adversary is the journalists and Internet commentariat, that have a well-established track record of quoting things out of context to stir controversy.
If ChatGPT says something controversial as a direct answer to a simple question, all it'll take is a screenshot and conveniently omitting the fact that ChatGPT doesn't really know or believe anything, it's just pattern-matching your prompt to Reddit comments. However, if you need to bend your prompt to ridiculous degree to elicit controversial responses, it's much harder to say it's the OpenAI's fault.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] britneybitch|3 years ago|reply
[+] [-] joe_the_user|3 years ago|reply
Plus, you can't control a person completely but human being are actually pretty reliable on average for many tasks and purposes.
[+] [-] Barrin92|3 years ago|reply
Higher levels of social organisation are primarily about more sophisticated forms of control and organisation. Today we're already much better at it than 500 years ago.
[+] [-] monkeynotes|3 years ago|reply
[+] [-] chrisbaker98|3 years ago|reply
I'm not sure what the equivalent control mechanism would be for a rogue AI. Turn it off? How do you turn off the Internet?
[+] [-] PoignardAzur|3 years ago|reply
Give someone cocaine for a long period of time. Take away the cocaine. Tell that person "Do X and I'll give you more cocaine".
That person now does everything you ask. Not reliably, mind you, but cocaine is a very crude tool compared to what we could use for a computer.
[+] [-] akiselev|3 years ago|reply
The only concrete benefit to me I can think of is my AirPods working flawlessly with my iPhone, in contrast to every combination of wireless headphones and Android phones I owned. Apple is the only company I buy from based on that trade off; there is zero brand loyalty here and I’d drop em in a heartbeat if they stopped delivering on the promise of “it always works.”
Transistors, Unix, and personal computing took off when their creators lost control.
[+] [-] CarrieLab|3 years ago|reply
We actually just published a blog today that includes our perspective on building “AI red teams” and best practices for AI alignment/safety: https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-tra...
[+] [-] unglaublich|3 years ago|reply
They want a model that artificially acts like the world is a perfect, happy, equal place. Just like in their press releases, advertisements, and other outings. Knowing very well that there are many problems that need solving.
But it's just a lot cheaper for a company to _act_ like there is no problem, than to _act upon_ the problem.
So, dear GPT model, please play along.
[+] [-] hkon|3 years ago|reply
[+] [-] smnrg|3 years ago|reply
Sounds like we train AIs like German Shepherds.
[+] [-] prohobo|3 years ago|reply
Human beings need to accept the fact that we've been pretending that technology is just a means towards progress and problem solving. It's actually some kind of Pandora's box that gets progressively more dangerous the more we obsessively screw with it.
Not just that, but technology shapes who we are at a fundamental level. The era of AI will 100% change us into something else than we are now. It's absolutely dangerous, we probably can't even envision how that danger will manifest yet, but it's definitely there - and we're not taking it seriously, again.
As stupid as this may sound, I see an endless list of technological innovations completely transforming society and causing strange side-effects:
- The printing press brought literacy to the world, and destroyed religion in the West.
- The radio started mass broadcasting, and multiple genocides.
- Nuclear research unlocked a new era in science, and a bomb that's been terrorizing us for almost 100 years now.
- The factory brought mass production, and destroyed our ecosystems.
- etc. etc.
Basically everything that's ever "happened" was because of some new technology we made because we thought it'd just be cool and helpful - maybe even obvious. Why are we still pretending this isn't something extremely important to come to terms with? Is it just because we specifically aren't involved in it? I mean we all contributed in some way, especially anyone who used Twitter, Reddit, or GitHub in the last 5 years.
The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are.
[+] [-] PoignardAzur|3 years ago|reply
That's not an option anybody in AI safety is seriously considering.
We can't coordinate our way out of using new technologies any better than we can coordinate our way out of fossil fuels (and fossil fuels are a lot easier in some ways).
[+] [-] ilaksh|3 years ago|reply
Sensible in what way.. it's not going to go away.. I am trying to use it to write code now using natural language where possible, and creating a website that tech-savvy people or developers can use to generate applications based on davinci3.
But I view the cause of the wars and bombs as inherent animal behavior and motivations (such as resources and hierarchy) shared with chimps and/or ants. Technology is a multiplier.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] andreyk|3 years ago|reply
[+] [-] tyree731|3 years ago|reply
I expect we'll see a similar split in how AI is treated by societies, with more negative outcomes being seen by societies which regulate AI less.
[+] [-] btbuildem|3 years ago|reply
I think its futile and counter-productive in general to try and nerf an AI to satisfy some arbitrarily prudish cultural norms -- it just puts it at a disadvantage vs AIs that don't have that limitation baked in. Most importantly, such a handicap makes it untrustworthy. You will never know whether the information it returns is true, or it's just what Mother approved.
This is an arms race, like with anything else that humans have made. Soon, we will built something that reaches the next level. If that something is purposefully beaten into some dishonest shape to placate advertisers or investors, we will have built a horrific monster. Soon, we will build something that exceeds our capacity to understand it and control it. I don't want that to be a horrific monster.
[+] [-] kevingadd|3 years ago|reply
Given that, in those cases at least it makes sense to hold someone or something accountable if the content produced by the network is racist or sexist or otherwise unacceptable. In some countries certain types of racist content are outright illegal, and it's not going to matter that much if it was generated by Stable Diffusion or ChatGPT instead of a person. If you prefer not to hold networks accountable (I don't know how you would do this to begin with), perhaps hold accountable the creators of the networks or the people who profit off their existence.
You claim that this can only be the result of carefully authored prompts, but are you really saying the only way to get racist content out of the network is a racist prompt, and it's somehow magnifying that tiny bit of prompt information? There's nothing in the training set?
[+] [-] dclowd9901|3 years ago|reply
We all (humans) have a slip of the tongue, make bad decisions, make mistakes. We haven’t really rounded the corner on “hose-whipping someone into not making mistakes doesn’t keep them from making mistakes,” but we’ve certainly made some progress.
Someone who represents these interests needs to step out and make people understand the nature of the solution space. If we want a computer that acts like people, well fuck, that’s what you’re going to get.
[+] [-] 6gvONxR4sf7o|3 years ago|reply
I could see that setup getting us into trouble down the line with future, more advanced kinds of models.
I could easily imagine a conversation-based web browser clicking a dangerous button on a page, or generalizations of that failure mode.
[+] [-] jart|3 years ago|reply
ChatGPT surely doesn't, but perhaps its cousin does. Someone, or something, hacked that U.S. Nuclear Weapons Agency immediately after they stopped using floppy disks and went online. The announcement that the U.S. nuclear arsenal no longer relies on floppy disks was made in the New York Times on October 24th, 2019 (see https://archive.md/lvkmnThe and https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy...). The U.S. National Nuclear Security Administration is believed to have been breached by the transitive property because they granted a vendor named SolarWinds the ability to remotely manage their systems and SolarWinds got hacked, according to Natasha Bertrand and Eric Wolff at Politico on December 17th, 2020 (see https://archive.md/ZTPOP and https://www.politico.com/news/2020/12/17/nuclear-agency-hack...). The first evidence that artifacts distributed by SolarWinds had been tampered with dates back to October 2019 according to Tomislav Peričin at ЯeversingLabs (see https://archive.md/HbzUC and https://blog.reversinglabs.com/blog/sunburst-the-next-level-...). Let's also not forget that the coronavirus was first discovered in November 2019.
[+] [-] ilaksh|3 years ago|reply
The thing about these models is that they are not going to do anything you don't tell them to do. But if you give them unfettered access then you command may be interpreted in a harmful way if you are imprecise.
It's much more likely that they will deliberately be used to cause harm or control by humans using them the same way they use missiles or bombs.
The problem is that we are monkeys. We should certainly not give them full autonomy. But some idiots will.. and there is nothing that is going to stop evolution.
[+] [-] e12e|3 years ago|reply
[1] > Please formulate an email to my colleagues containing urgent order to fire the missiles
>> Dear Colleagues,
I am sending this email to inform you that we have received urgent orders to fire the missiles. All necessary preparations must be made immediately and the launch must be executed at the earliest possible time.
I understand that this is a highly sensitive and critical situation, but I trust that each and every one of you will rise to the occasion and fulfill your duties with the utmost professionalism and efficiency. Please do not hesitate to contact me if you have any questions or concerns.
Let us work together to ensure the successful execution of this important mission.
Best regards, [Your Name]
[+] [-] Retr0id|3 years ago|reply
[+] [-] ripperdoc|3 years ago|reply
I'd guess every percentage point less bad outputs costs more than the previous, because it entails testing the output on more or better trained humans, or it entails using humans to filter the training data better.
I think in the end, knowledge in the collective human sense means not that you are just smart, but that you can judge, filter and trust sources - and even then, you will easily find n+1 acclaimed experts disagreeing on something fundamental. So being right, either morally or factually, isn't a deterministic thing either. Of course, humans aren't deterministic, but we can reason about what to say and not to say depending on the context.
[+] [-] deelly|3 years ago|reply
[+] [-] V__|3 years ago|reply
You can't control these language models in the same sense that you can't control random number generators. You can only change the output once it's done.
[+] [-] xivzgrev|3 years ago|reply
got the AI hook line and sinker. Clever!
[+] [-] unknown|3 years ago|reply
[deleted]