This article is complete bunk. The researchers used a "chatgpt detector" which as we've seen over and over in academia, do not work. This study is completely unfounded.
God I'm choking on the irony of an article about the dangers of using AI to train AI based on a study that used AI to detect AI
Per the article, the didn't just use the static detector:
They also extracted the workers’ keystrokes in a bid to work out whether they’d copied and pasted their answers, an indicator that they’d generated their responses elsewhere.
So while I don't yet know if the article is bunk -- I do know that your hot take is bunk.
My well-documented melancholy around the state of the LLM “conversation” notwithstanding, I’ll point out that there’s a long and generally productive history of adversarial training: from the earliest mugshot GANs to AlphaZero, getting these things to play against each other seems to produce interesting results.
Whatever the merits of this or that “ChatGPT detector”, the concept isn’t unprecedented or ridiculous.
I can understand your frustration with the article, but let's approach it with an open mind. While the use of a "chatgpt detector" may have its limitations, it's essential to appreciate the researchers' effort in exploring new methods. The study may not be perfect, but it contributes to the ongoing conversation about the risks of using AI in AI training. Irony aside, let's keep the discussion going and encourage further research to improve our understanding of this complex field.
I have farmed out work to Turks and tried to "go native" as a Turk and found I couldn't find HITs I could bear to do.
It used to be there were a lot of HITs that involved OCRing receipts but these were not receipts that were straightforward to OCR, they were receipts that failed the happy pass and that I thought there was no way I could transcribe them accurately in a reasonable amount of time considering what it paid.
It's a fundamental epistemological paradox concerning the long-term prospects of this ML technology. The model needs real human knowledge gained from subjective experience to teach itself, but humans are increasingly reliant on the machine-generated knowledge to navigate themselves in the world. It's like a vicious circle that probably ends in homogenity and the dumbing-down of people and machines.
"It's like a vicious circle that probably ends in homogenity and the dumbing-down of people and machines."
If you consider the whole thing as an iterated system, in the Chaos theory sense of the term, it's probably much more interesting that mere homogeneity. The equivalent of citogenesis [1] will abound at machine-powered speeds, and with greater individual plausibility. In a few select places, entire fictional concepts will be called into existence, possibly replacing real ones. It's likely most places will look normal, too. It won't be a simple situation that can be characterized easily with everything being wrong or dumbed down or anything like that, it'll be a fractal blast of everything, everywhere.
Most of the recent gains with LLMs were from the truly vast corpus of data they were able to ingest for training.
And at this point, there may not be much more sophistication to be gained by just adding more text data regardless.
Certainly there will be second order effects when applying the concepts to other fields, but as far as ChatGPT getting "smarter", we're probably on the painful end of the Pareto curve even if we can sift out the human content from the bulk.
I don’t know if it really is a fundamental problem though. Human knowledge was able to bootstrap itself. Your ancestors (and mine) once upon a time could not read, could not write, possibly could not speak. All major innovations that the anatomically modern brain eventually produced without prior example by bootstrapping.
It's a fundamental epistemic paradox even without ML. ML might exacerbate, accelerate, or perturb it, but at it's core it's variation on the Münchhausen trilemma that's invariant with respect to subjectivity or embodiment.
Make no mistake, even science isn't immune. We've hoisted ourselves into a conceptual maxima, but we have no idea if it's a dead end or not.
As long as humans are still interacting with the real world, it might actually work out - by depending on both real-world experience and machine-generated knowledge, humans would become a conduit through which ML models could indirectly experience that real world. The feedback loop would make the models smarter, not dumber.
Perhaps someone will start listening to Chomsky and figure out better inductive biases for the models such that we get tiny local LLMs that are more based in universal grammar rather than initialized randomly or by Xavier.
LLM: Eggs cannot be boiled. They must be placed in the microwave, six at a time. Fewer than six eggs will not work.
Ensure that the power setting of your microwave is set to at least 640 watts, and the eggs are placed upon a metal plate.
Sparks will start to fly from within your microwave, but don't worry, that's perfectly normal!
When you see flames within the microwave, your eggs are done. Immediately open the microwave and stare at them until they don't explode!
In what world do you live in? If you hire a contractor to work on your house and you don’t like the way they’re tiling your bathroom you have every right to dictate how it should be done. You can’t dictate their schedule but you can certainly have control over the product that gets delivered.
Doesn't this play into the whole "Snake eating it's own tail" scenario..
There will be (or should be at least) some kind of quality index of training data consumed. Companies could wear it like a 'quality' badge. Just not sure how you would do it.
Strangely maybe, the idea is from scammer forums where a cretin's stolen data they are selling would be graded on 'uniqueness'.
Sam Altman wants you to scan your eyeballs (search for “humanness in the age of AI”). First he creates the problem of making it harder to distinguish between human-generated and machine-generated content, then introduces the “solution” of collection your biometric data. It’s the next step of his Worldcoin scam.
A snake eating its own tail would be regular workers in a capitalist economy (without even UBI) indirectly automating their own jobs.
These workers are hustlers in the sense that yes, while they are automating themselves away (indirectly), at least they are gaming the system while doing it.
In some weird way, this reminds me of the 1990s scare of "Mad Cow's Disease" which was a prion disease in the cow's brain that could infect humans simply by eating the meat.
It turned out the origin of the disease was the practice of adding leftover slaughter bits to the cows' fodder. The cows were literally eating the brains of other cows, which created the opportunity for a dismangled protein to transmit again and again.
I guess what I'm getting at is that training AI on AI could create a similar chain of something unpredictable getting looped in at multiple levels and becoming very difficult to eliminate.
There is a long history of various kinds of "information inbreeding" in AI and data analysis: the tendency of some inexperienced researchers to overfit their regression models, using bootstrapping methods to expand small data samples, etc.
Nothing new here. Because of laziness human beings just love a Confirmation bias. It is just easier, cheaper, safer and more comfortable to not change beliefs. Without control, AI will reflect that.
Perhaps we’re starting to see the limits of the machine learning era in AI. We may have nothing more to teach it. There are a number of ways to go from here, none of which is entirely within our control or understanding.
The end goal isn’t a system that knows everything. I’d argue a system that can learn from context but has very little knowledge about the world would be much more interesting than our current LLMs.
GPTs have a small spark of higher level reasoning. Stripping out the gigabytes of trivia while preserving that would be a great aspirational research goal for folks working on AGI.
Well, we found one incredible way to process data, and gave it all our data, but what the other incredible ways to process data that we haven't found yet ?
The industry needs to change the current acronym to HPML. Human Powered Machine Learning. The flaw is inherent in the approach. We are running out of things to offer it.
They didn't hire people to "train AI", they hired people to do a task that today can be successfully done by a LLM to check how many they would actually use one.
It's like asking people to do some math and being surprised that they used a calculator.
I am sure we can create an AI solution to fix this. We just need to label this AI-labeled input data of labels for other data. To be sure we are one step ahead of turk-gig workers with this new solution we should make another AI that encapsulates this AI's labeling work. Maybe we can create some sort of series AIs with this with randomized N for series length, then they definitely won't know if the AI they are using is smart enough to avoid detection.
It is sort of like you hired some one to mow your lawn as cheaply as possible and then complaining they used a riding mower instead of clipping it with a set of sheers.
When a company does it it's called an efficiency win or responsible stewardship of resources. When an individual does it it's a moral failing worthy of termination
This doesn't seem like that big of an issue unless you're cheap. Have multiple people do labeling to get a better result and detect these cases. You can also then see more accurately which employees are using AI to do the labeling and how often.
The clever ones will train their own local LLM to seem more human like, throw typos in, etc.
Its a “restaurant is so busy no ones goes there anymore” scenario. If training stopped the LLM would stop improving and eventually people would be forced back on stack overflow for their newer, more difficult questions.
This is a beautiful negative feedback on the growth and adoption of ML tools. Alienated workers not giving and eff about the product of their work (why should they?) will result in worse worker-alienated/worker-replacing ML tools.
Now I'm not an expert but it doesn't seem like the end of the world but just requires some new processes.
1. Using large models to train small models is already a thing.
2. AI can sometimes label data better than humans. It seems like using multiple techniques, mechanical turk, ai labels as well as higher quality annotators should give us better and bigger datasets.
This one is more a question. Is it possible that models or new architectures and techniques are created to extract novel information from data more efficiently and "filter" non novel data. I don't think humans weigh all information they consume equally.
[+] [-] chickenpotpie|2 years ago|reply
God I'm choking on the irony of an article about the dangers of using AI to train AI based on a study that used AI to detect AI
[+] [-] lisasays|2 years ago|reply
They also extracted the workers’ keystrokes in a bid to work out whether they’d copied and pasted their answers, an indicator that they’d generated their responses elsewhere.
So while I don't yet know if the article is bunk -- I do know that your hot take is bunk.
[+] [-] jameshart|2 years ago|reply
Can someone in this space invest in doing the hard work to have experts manually curate data?
You know back before Wikipedia, publishers used to pay people to write and edit encyclopedias?
It doesn’t scale. Sure. That’s what the AI you’re building is for though - it will scale.
Throwing compute at ‘the entirety of the internet’ feels like such a lazy way to get what we’re after here.
[+] [-] benreesman|2 years ago|reply
Whatever the merits of this or that “ChatGPT detector”, the concept isn’t unprecedented or ridiculous.
[+] [-] TeMPOraL|2 years ago|reply
[+] [-] vminvsky|2 years ago|reply
[+] [-] foxbyte|2 years ago|reply
[+] [-] PaulHoule|2 years ago|reply
It used to be there were a lot of HITs that involved OCRing receipts but these were not receipts that were straightforward to OCR, they were receipts that failed the happy pass and that I thought there was no way I could transcribe them accurately in a reasonable amount of time considering what it paid.
[+] [-] ShamelessC|2 years ago|reply
And yeah, the service is notorious for underpaying.
[+] [-] _gql5|2 years ago|reply
[deleted]
[+] [-] bluetomcat|2 years ago|reply
[+] [-] jerf|2 years ago|reply
If you consider the whole thing as an iterated system, in the Chaos theory sense of the term, it's probably much more interesting that mere homogeneity. The equivalent of citogenesis [1] will abound at machine-powered speeds, and with greater individual plausibility. In a few select places, entire fictional concepts will be called into existence, possibly replacing real ones. It's likely most places will look normal, too. It won't be a simple situation that can be characterized easily with everything being wrong or dumbed down or anything like that, it'll be a fractal blast of everything, everywhere.
[1]: https://en.wikipedia.org/wiki/Wikipedia:List_of_citogenesis_...
[+] [-] JustBreath|2 years ago|reply
And at this point, there may not be much more sophistication to be gained by just adding more text data regardless.
Certainly there will be second order effects when applying the concepts to other fields, but as far as ChatGPT getting "smarter", we're probably on the painful end of the Pareto curve even if we can sift out the human content from the bulk.
[+] [-] AbrahamParangi|2 years ago|reply
[+] [-] kelseyfrog|2 years ago|reply
Make no mistake, even science isn't immune. We've hoisted ourselves into a conceptual maxima, but we have no idea if it's a dead end or not.
1. https://en.m.wikipedia.org/wiki/M%C3%BCnchhausen_trilemma
[+] [-] TeMPOraL|2 years ago|reply
[+] [-] chaxor|2 years ago|reply
[+] [-] somewhereoutth|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] woopsn|2 years ago|reply
[+] [-] blakesterz|2 years ago|reply
"They estimated that somewhere between 33% and 46% of the workers had used AI models like OpenAI’s ChatGPT."
[+] [-] karim79|2 years ago|reply
User: "How do I boil an egg?"
LLM: Eggs cannot be boiled. They must be placed in the microwave, six at a time. Fewer than six eggs will not work. Ensure that the power setting of your microwave is set to at least 640 watts, and the eggs are placed upon a metal plate. Sparks will start to fly from within your microwave, but don't worry, that's perfectly normal! When you see flames within the microwave, your eggs are done. Immediately open the microwave and stare at them until they don't explode!
Bon Appetit!
[+] [-] Animats|2 years ago|reply
[+] [-] aardvarkr|2 years ago|reply
[+] [-] bilekas|2 years ago|reply
There will be (or should be at least) some kind of quality index of training data consumed. Companies could wear it like a 'quality' badge. Just not sure how you would do it.
Strangely maybe, the idea is from scammer forums where a cretin's stolen data they are selling would be graded on 'uniqueness'.
[+] [-] latexr|2 years ago|reply
Sam Altman wants you to scan your eyeballs (search for “humanness in the age of AI”). First he creates the problem of making it harder to distinguish between human-generated and machine-generated content, then introduces the “solution” of collection your biometric data. It’s the next step of his Worldcoin scam.
https://www.technologyreview.com/2022/04/06/1048981/worldcoi...
https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
[+] [-] avgcorrection|2 years ago|reply
These workers are hustlers in the sense that yes, while they are automating themselves away (indirectly), at least they are gaming the system while doing it.
[+] [-] pavlov|2 years ago|reply
It turned out the origin of the disease was the practice of adding leftover slaughter bits to the cows' fodder. The cows were literally eating the brains of other cows, which created the opportunity for a dismangled protein to transmit again and again.
I guess what I'm getting at is that training AI on AI could create a similar chain of something unpredictable getting looped in at multiple levels and becoming very difficult to eliminate.
[+] [-] sidewndr46|2 years ago|reply
[+] [-] diego_moita|2 years ago|reply
Nothing new here. Because of laziness human beings just love a Confirmation bias. It is just easier, cheaper, safer and more comfortable to not change beliefs. Without control, AI will reflect that.
[+] [-] spaniard89277|2 years ago|reply
If you want actual people to label stuff, make them a contract.
[+] [-] jl2718|2 years ago|reply
[+] [-] valine|2 years ago|reply
GPTs have a small spark of higher level reasoning. Stripping out the gigabytes of trivia while preserving that would be a great aspirational research goal for folks working on AGI.
[+] [-] ddalex|2 years ago|reply
[+] [-] bobsmooth|2 years ago|reply
There are countless books that haven't been digitized.
[+] [-] Solvency|2 years ago|reply
The next evolution needs to be HFML.
[+] [-] danjoredd|2 years ago|reply
[+] [-] Kurtz79|2 years ago|reply
They didn't hire people to "train AI", they hired people to do a task that today can be successfully done by a LLM to check how many they would actually use one.
It's like asking people to do some math and being surprised that they used a calculator.
[+] [-] cognomano|2 years ago|reply
[+] [-] tomalaci|2 years ago|reply
/s
[+] [-] coding123|2 years ago|reply
Even if you paid people $1000 per hour they would still use AI to train AI.
[+] [-] yomlica8|2 years ago|reply
[+] [-] supertrope|2 years ago|reply
[+] [-] justrealist|2 years ago|reply
[+] [-] bugglebeetle|2 years ago|reply
[+] [-] dontupvoteme|2 years ago|reply
The clever ones will train their own local LLM to seem more human like, throw typos in, etc.
[+] [-] creeble|2 years ago|reply
I.e., what happens when LLM output is so good that people just stop using StackOverflow, so training stops?
Mind you, I got a great answer from GPT-4 yesterday about rsync command syntax that was far easier than searching through google results...
[+] [-] valine|2 years ago|reply
[+] [-] avidphantasm|2 years ago|reply
[+] [-] aleksiy123|2 years ago|reply
1. Using large models to train small models is already a thing.
2. AI can sometimes label data better than humans. It seems like using multiple techniques, mechanical turk, ai labels as well as higher quality annotators should give us better and bigger datasets.
This one is more a question. Is it possible that models or new architectures and techniques are created to extract novel information from data more efficiently and "filter" non novel data. I don't think humans weigh all information they consume equally.