If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life.
This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal.
ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further.
All signs point to this strengthening the value of curation and authenticated sources.
The IP resolves to Digital Ocean. So I doubt it's on their level. The author themself probably saw the server melting down and/or hosting bill going up and threw up a really silly "solution".
Had this happen to me too. For a second, I was wondering if the title of this HN post and the Google redirect was some sort of joke I didn't understand.
Internet often feels it's getting rotten with SEO spam with very little substance already. Now the flood gates are open to generate more shit in a much better wrapping. And it's going to take more effort to check its quality than to produce it. Unless we get better tools to do that - but text has more dimensions to process/understand as opposed to for example AI generated images which (for now) can be told apart by glitches, or trails detectable by software.
What was the saying? It takes 10x more effort to refute BS than it takes to produce it? So if we make generating BS easier and that BS is fed for generating more BS we're going to get to a point where everything is BS. I call this the BS singularity!
I think an easy way to identify transformative technology is how strong people’s reaction is against it. I remember similar freak outs about the graphical World Wide Web, smart phone, etc. Somehow things are not quite so bad and considerably better than predicted, while still having negative side effects.
Maybe an outcome will be knowledge will be better structured, ala Wolfram Alpha, and just depending on random text documents to encode knowledge won’t be as much a thing - similar to how we no longer use oral traditions to encode and disseminate knowledge. Who knows. But I doubt AI will fundamentally destroy society or knowledge or the internet or whatever. We will adapt, not just to preserve the way we did things, but to use the new things in a way that controls its side effects. It’ll of course be imperfect and some side effects can’t be controlled for. Some things just will cease to be a thing, like phone booths. While you can never have a bill and teds with a smartphone, it’s actually not the end of things. It’s just different. That’s how it works my friend. Life is impermanent, everything changes at all times, you can never recreate the past, and our suffering stems from our inability to let go of the way things are.
Edit: I hate to bring it up, but maybe this is what the semantic web was waiting for.
> I think an easy way to identify transformative technology is how strong people’s reaction is against it.
There is also strong negative reaction to technology that does incredible damage to society. This is just an excuse to ignore people pointing out issues.
I think this will push society toward better solutions to human verification. Probably through governments or their corporate proxies. Will that end up good or bad? I have no idea, but we'll find out soon.
This is the only semi-plausible positive take I've seen on ChatGPT. If we actually manage to move the world towards storing, presenting, and consuming facts in well-defined formats rather than long-form prose, that could possibly be an improvement on the current situation.
So my first take is that people querying it for research are doing it wrong.
Then again, if there’s a large economic incentive to use it in that way, we are very well may end up with the kind of feedback loop that the author describes.
You’re absolutely right on all counts, and yet, people Are doing it wrong, and will increasingly do so, because there Are large economic incentives for using it that way.
Behind this thinking - an economic incentive to access knowledge and answers quicker that is inevitable - we have built a tool that verifies statements using source materials. You have to put the legwork in to upload your PDFs/web pages/videos, but once you do, you can be confident in the answers.
If it can't verify, it just won't answer/tickmark check the answers (happens 16% of the time... and ... always for maths). This is a feedback loop stopper, in the sense of only relying on your documents as the base, and being able to operate entirely without OpenAI (still though using other GPT models)
It's Fragen.co.uk - we believe that more answers formerly missed by CTRL+F will be found with this technology, than false answers taken as true. And if that's true, you are enbettering knowledge. And if not, you're enshittening it slower than the higher-hallucinating alternatives.
The internet has been groaning under the load of bots, automation and distrust for a while now anyway: in my mind what is needed is a system for proving constituent parts of your identity which is also privacy preserving. I'm thinking of the work around DID re: rebooting the web of trust, and the W3C working group
If you can wield an ephemeral and verifiable token which asserts your humanness, hierarchically derived from a certificate privately issued to you by one of hopefully a healthy number of well known authorities, you can participate within circles of the human web without revealing anything else about your identity (name, a/s/l, etc)
But, outside of this enclave you can also interact with AI without revealing whether in fact you too are AI.
In this way the internet can develop a more sensitive immune system where it is difficult for human systems to be perverted by Sybil attacks.
There's a famous quotes from Bill Gates that is "My children will have computers, yes, but before they will have books. Without books, without reading, our children will be unable to write, including their own story."
On my personal experience, ChatGPT is a precious tool to refine knowledge and thinking, but it depends on how it's used.
The same painting brush can be used to paint Monalisa or a meaningless blurred thing. It depends on how it's used.
To use ChatGPT and other AI tools properly, we need to build a solid foundation of knowledge and critical thinking skills at first.
Then triangulate the answers given by the AI with other sources, our own previous knowledge along with new evidences through experimentation.
Since the brain urges for cognitive relief, most people will be lazy, while just a few will extract the true potential of ChatGPT.
> Therefore, our safeguard is constrained by time, knowledge, and resources. The problem then is that if that safeguard fails and A-Grade Bullshit is taken by an author using the AI as a research assistant, put into a research paper, book chapter, or online article, and is then cited. That becomes potential training data for the AI if it is released into the world or has the window of its training data set extended.
I don't see the value in GPT models for this kind of use. I see the value in like, translation work, or summaries and other transformations, and that GPT works best when in concert with search engines, document embeddings, or knowledge databases like Wolfram Alpha. It's good at predicting more text which is useful in many different kinds of contexts but not in all contexts. It's pretty clear that it is untrustworthy for factual responses like quotes and population counts but that problem goes away if the actual factual data is included in the prompt!
Isn't that where ChatGPT would actually be worse overall? It's been shown to give semi-accurate data very often while sounding very confident about it.
When people require precise answers and it gives almost correct answer, it seems the general feeling is of amazement. I don't share that feeling. I hope nobody is using it for serious work without a human vetting the output.
Maybe I'm being too simplistic, but I think going forward critical thinking as well as an understanding of what to value is going to be important for future generations. Something that computers will never be good at doing is 'understanding' what is valuable in life since many times this cannot be measured. In essence, a healthy understanding of the three transcendentals the Good the True, and the Beautiful will be essential to keep our humanity. ChatGPT can perform many amazing skills but lacks the ability to accurately judge the value of what is it reporting. I think passing on a love for the transcendentals to our children is the only way to inoculate them from the firehose of 'junk' information ChatGP et al is about to unleash on the world.
Think of it like this. AI is the Mandela Effect made real.
If the preponderance of human writing/creation/art says the sun revolves around the earth, then AI not only nails that down as infallible, but INFERS from it and comes up with new justification for stuff that it made up.
Mandela Effect is all about what 'should' be the truth. Some of the most interesting parts of reality are where bits of reality don't neatly fit into the narrative.
AI exploiting its own inferences and magnifying the Mandela Effect means it steamrollers inconvenient realities on the grounds that 'most people' wouldn't think so. But we don't even think of AI as 'most people', we imagine it as some kind of super-set of humanity, the ultimately wise and skilled overseer.
> When we turn the job of creating the first draft of the thing over to ChatGPT we risk removing the “figuring shit out” part of everyone’s career-path. And we get away with that while we still have people who have figured shit out.
Agreed. The LAST use of ChatGPT should be the first draft. It's clueless about substance or the purpose of the essay, neither what's necessary nor essential. As such, its best use is to flesh out an outline that HAS these things into a essay or narrative that integrates them into a narrative that flows.
The trick to that is, how do you take away ChatGPT's inclination to build upon the wrong premises or the wrong final message, and instead confine it to the role of verbally adroit assistant -- so it only expands a skeletal design into an well written final product? Now THAT would be a use for ChatGPT I could get behind.
Alas, that wasn't the goal of its designers. By giving it almost total control over the plot line and requiring no fact checks or bibliography, the spewing of unchecked drivel was its only possible mission in life.
I see chatGPT as a first iteration, and it’s glaringly obvious what works and what doesn’t. I’m hopeful future iterations (not just from openAI) will make it overall a net improvement, things like adding real sources and better handling things it doesn’t know
The combination of using an "authoritative" voice and being completely and utterly full of shit makes for some truly ridiculous results. Users need to be extremely careful with these tools. And no, expecting everyone to be knowledgeable to independently fact check some BS published based on a generative model is not a reasonable position. That burden needs to be held by the publisher.
Unfortunately with the current state of machine generated content and SEO spam on the internet, that is not likely to happen as there is no real incentive to do so.
And it's not just being confidently wrong about a single person like this author noted, it's being confidently wrong about anything and everything.
The other day I asked ChatGPT about the reasons behind the connection between Italian filmmakers and Western movies. It gave a few reasons that were semi-plausible if not easily verifiable, and then made the bold claim that Italy and the United States were great allies in World War 2. It said this confidently and with a tone of authority.
And while we may all go "ha ha that's wrong" now, WW2 was nearly 80 years ago and not everyone has a strong history background. There aren't that many people left with direct experience and that number shrinks every day. With growing distrust in actual authorities on matters and rejection of observable fact--based at least in part on massive piles of internet BS--this is pretty concerning to me.
The test use case of constructing a bio for yourself, hoping it accurately summarizes all the extremely low sample size data it happens to have of you in its web crawled training data, seems like one of the worst possible use cases for ChatGPT. It’s right there on the main page that it’s not to be trusted with factual information like this. ChatGPT will hallucinate details. It’s remarkable to me actually how often it will refuse to hallucinate, given that’s basically what its job is. I don’t find it interesting to find all these edge cases where ChatGPT produces empirically false data. It doesn’t even have the ability to look things up! If I were the OP and wanted help writing my bio, I would first write the draft myself, then use ChatGPT to help with the editing, prose, grammar, style, etc. You are the expert on the factual details of your own life, and if you’re surprised that a language model trained on web crawled data ending in 2018 is not, then all I’ve learned is that you don’t know much about what this thing is.
I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook.
ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)
I feel like the 3 youre missing there is something along the lines of "people enjoy social validation and internet points, to the extent that pretty shit content thats low effort is something we enjoy generating"
Prediction: OpenAI/GPT (or Google's DeepMind, if you prefer) is going to cause mass unemployment in certain sectors of the economy (for example, graphic designers, copywriters, and many IT professionals) long before it ever addresses any of the fundamental problems that currently substantially reduce human quality and quantity of life, like aging or cancer.
In the near-term, AI will just accelerate the winner-take-all nature of our economy.
[+] [-] lubujackson|3 years ago|reply
This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal.
ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further.
All signs point to this strengthening the value of curation and authenticated sources.
[+] [-] voytec|3 years ago|reply
[+] [-] rkimb|3 years ago|reply
[+] [-] jdwithit|3 years ago|reply
[+] [-] jabo|3 years ago|reply
[+] [-] x86x87|3 years ago|reply
[+] [-] bartread|3 years ago|reply
[+] [-] posix86|3 years ago|reply
[+] [-] bcjordan|3 years ago|reply
[+] [-] jagged-chisel|3 years ago|reply
[+] [-] pawelduda|3 years ago|reply
[+] [-] x86x87|3 years ago|reply
[+] [-] orangepurple|3 years ago|reply
[+] [-] TSUTiger|3 years ago|reply
[+] [-] fnordpiglet|3 years ago|reply
Maybe an outcome will be knowledge will be better structured, ala Wolfram Alpha, and just depending on random text documents to encode knowledge won’t be as much a thing - similar to how we no longer use oral traditions to encode and disseminate knowledge. Who knows. But I doubt AI will fundamentally destroy society or knowledge or the internet or whatever. We will adapt, not just to preserve the way we did things, but to use the new things in a way that controls its side effects. It’ll of course be imperfect and some side effects can’t be controlled for. Some things just will cease to be a thing, like phone booths. While you can never have a bill and teds with a smartphone, it’s actually not the end of things. It’s just different. That’s how it works my friend. Life is impermanent, everything changes at all times, you can never recreate the past, and our suffering stems from our inability to let go of the way things are.
Edit: I hate to bring it up, but maybe this is what the semantic web was waiting for.
[+] [-] banannaise|3 years ago|reply
There is also strong negative reaction to technology that does incredible damage to society. This is just an excuse to ignore people pointing out issues.
[+] [-] Method-X|3 years ago|reply
[+] [-] l0b0|3 years ago|reply
[+] [-] 93po|3 years ago|reply
[+] [-] clbrmbr|3 years ago|reply
So my first take is that people querying it for research are doing it wrong.
Then again, if there’s a large economic incentive to use it in that way, we are very well may end up with the kind of feedback loop that the author describes.
[+] [-] roughly|3 years ago|reply
[+] [-] james-revisoai|3 years ago|reply
If it can't verify, it just won't answer/tickmark check the answers (happens 16% of the time... and ... always for maths). This is a feedback loop stopper, in the sense of only relying on your documents as the base, and being able to operate entirely without OpenAI (still though using other GPT models)
It's Fragen.co.uk - we believe that more answers formerly missed by CTRL+F will be found with this technology, than false answers taken as true. And if that's true, you are enbettering knowledge. And if not, you're enshittening it slower than the higher-hallucinating alternatives.
[+] [-] manytree5|3 years ago|reply
If you can wield an ephemeral and verifiable token which asserts your humanness, hierarchically derived from a certificate privately issued to you by one of hopefully a healthy number of well known authorities, you can participate within circles of the human web without revealing anything else about your identity (name, a/s/l, etc)
But, outside of this enclave you can also interact with AI without revealing whether in fact you too are AI.
In this way the internet can develop a more sensitive immune system where it is difficult for human systems to be perverted by Sybil attacks.
[+] [-] brap|3 years ago|reply
[+] [-] fnfontana|3 years ago|reply
On my personal experience, ChatGPT is a precious tool to refine knowledge and thinking, but it depends on how it's used.
The same painting brush can be used to paint Monalisa or a meaningless blurred thing. It depends on how it's used.
To use ChatGPT and other AI tools properly, we need to build a solid foundation of knowledge and critical thinking skills at first. Then triangulate the answers given by the AI with other sources, our own previous knowledge along with new evidences through experimentation.
Since the brain urges for cognitive relief, most people will be lazy, while just a few will extract the true potential of ChatGPT.
[+] [-] erksa|3 years ago|reply
[+] [-] williamcotton|3 years ago|reply
I don't see the value in GPT models for this kind of use. I see the value in like, translation work, or summaries and other transformations, and that GPT works best when in concert with search engines, document embeddings, or knowledge databases like Wolfram Alpha. It's good at predicting more text which is useful in many different kinds of contexts but not in all contexts. It's pretty clear that it is untrustworthy for factual responses like quotes and population counts but that problem goes away if the actual factual data is included in the prompt!
[+] [-] gtirloni|3 years ago|reply
Isn't that where ChatGPT would actually be worse overall? It's been shown to give semi-accurate data very often while sounding very confident about it.
When people require precise answers and it gives almost correct answer, it seems the general feeling is of amazement. I don't share that feeling. I hope nobody is using it for serious work without a human vetting the output.
[+] [-] AlbertCory|3 years ago|reply
Yesterday I asked it "what are the chords to 'A Foggy Day in London Town'": it made up a whole new Gershwin song!
======
"A Foggy Day in London Town" is a song with music by George Gershwin and lyrics by Ira Gershwin. The chords for the song are:
Verse:
C C7 F C C7 F C C7
Foggy, foggy dew, you can't see a thing
F Fm C C7 F C C7 F
The visibility is zero, and my eyes are dim
Chorus:
C C7 F C C7 F C C7
It's a foggy day in London Town
F Fm C C7 F C C7 F
The buildings look one hundred feet tall
C C7 F C C7 F C C7
It's a foggy day in London Town
F Fm C C7 F C C7 F
And the people look like ants at all
Bridge:
Am D7 G C C7 F C C7
Just like the feeling you get when you're down
Am D7 G C C7 F C C7
The sun can't break through the London Town
[+] [-] AlphaOne1|3 years ago|reply
[+] [-] Applejinx|3 years ago|reply
If the preponderance of human writing/creation/art says the sun revolves around the earth, then AI not only nails that down as infallible, but INFERS from it and comes up with new justification for stuff that it made up.
Mandela Effect is all about what 'should' be the truth. Some of the most interesting parts of reality are where bits of reality don't neatly fit into the narrative.
AI exploiting its own inferences and magnifying the Mandela Effect means it steamrollers inconvenient realities on the grounds that 'most people' wouldn't think so. But we don't even think of AI as 'most people', we imagine it as some kind of super-set of humanity, the ultimately wise and skilled overseer.
Boy, is that a mistake.
[+] [-] discreteevent|3 years ago|reply
That's the main problem from my point of view.
[+] [-] randcraw|3 years ago|reply
The trick to that is, how do you take away ChatGPT's inclination to build upon the wrong premises or the wrong final message, and instead confine it to the role of verbally adroit assistant -- so it only expands a skeletal design into an well written final product? Now THAT would be a use for ChatGPT I could get behind.
Alas, that wasn't the goal of its designers. By giving it almost total control over the plot line and requiring no fact checks or bibliography, the spewing of unchecked drivel was its only possible mission in life.
Congrats, OpenAI. You've automated spam.
[+] [-] visarga|3 years ago|reply
[+] [-] jerpint|3 years ago|reply
[+] [-] DebtDeflation|3 years ago|reply
Archive link: https://web.archive.org/web/20230127104933/https://castlebri...
[+] [-] scblock|3 years ago|reply
Unfortunately with the current state of machine generated content and SEO spam on the internet, that is not likely to happen as there is no real incentive to do so.
And it's not just being confidently wrong about a single person like this author noted, it's being confidently wrong about anything and everything.
The other day I asked ChatGPT about the reasons behind the connection between Italian filmmakers and Western movies. It gave a few reasons that were semi-plausible if not easily verifiable, and then made the bold claim that Italy and the United States were great allies in World War 2. It said this confidently and with a tone of authority.
And while we may all go "ha ha that's wrong" now, WW2 was nearly 80 years ago and not everyone has a strong history background. There aren't that many people left with direct experience and that number shrinks every day. With growing distrust in actual authorities on matters and rejection of observable fact--based at least in part on massive piles of internet BS--this is pretty concerning to me.
[+] [-] marban|3 years ago|reply
[+] [-] khiner|3 years ago|reply
I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook.
ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)
[+] [-] mkmk3|3 years ago|reply
[+] [-] throwaway589275|3 years ago|reply
In the near-term, AI will just accelerate the winner-take-all nature of our economy.