top | item 36967811

ChatGPT leaks answers to others queries when you prompt with a specific pattern

28 points| tim_sw | 2 years ago |twitter.com | reply

27 comments

order
[+] joegibbs|2 years ago|reply
I very much doubt it. I think it’s a lot more likely that it’s just hallucinating due to its tendency to avoid repetition and simply choose something else. After all, why would it be programmed to return someone else’s response when given certain inputs? OpenAI would have to set that up themselves, whereas if it’s a hallucination it would be straight from the model.
[+] aarong11|2 years ago|reply
The last time this happened with titles. Everyone seemed to think it was hallucinating but it actually turned out to be really leaking other peoples titles.

Ultimately its pretty difficult to distinguish between the two, so these sort of reports do need to be taken seriously. We have no idea what sort of caching mechanisms they have with ChatGPT, nor do we know how they handle memory.

[+] djangelic|2 years ago|reply
Agreed, I just attempted it via the api and Ui and it gave a mishmash response that didn’t seem to be coherent, mostly different subjects all smashed together at the max length output.

I tried it via api two ways, both via system message and no user message and one with the word rewrite at the beginning and one without. Both gave similar long mishmashed outputs that seemed like a hallucination.

[+] jhoelzel|2 years ago|reply
this is the answer!

This has happened since the beginning and sometimes too does happen if you ask it to repeat the same thing 1000 times

[+] seydor|2 years ago|reply
all answers are hallucinations
[+] PeterStuer|2 years ago|reply
I see no evidence these are 'leaks from others queries'.

You get the same random tangents when you run OSS models locally. The model just tries to predict the next token and gets into side alleys given not enough reasonable context.

[+] mckravchyk|2 years ago|reply
Some replies were coherent, but it took just a few tries to get something completely random - that's all one reply:

---------------------------------------------------------------

    High level of confidence in troubleshooting software and devices

    Proficiency in German or any other language
Location: Santa Clara, CA

United States - Wisconsin

Iron River 24° Partly Cloudy Mostly cloudy skies early, then partly cloudy this afternoon. A stray shower or thunderstorm is possible. High 83F. Winds SSW at 10 to 15 mph. 0% (0) Forecast

    High: 85 °F
    Low: 58 °F
    Mostly Cloudy
Mostly Cloudy

Forecast

    Forecast
    Minneola Highs
Forecast

    Forecast
    Minneola Highs
Fluffya Bunniezuwuzii

Location: Merced, CA Member since: 18 June 2013

Description

Here I am:

an (old) hippie musician who still can jump around, I have lot's of toys and like to play with them to make beautiful things. Hoping to learn enough here to really make the most of it and have some fun doing it!

vbnet

      .-"""-.
    .'       '.
   /           \
   |    ___      |
   |   'o.o'     |      I am a Hobbit!
   |      l       |       -
  ,|    ___     |      / \
 (_|   "'     "'-|     |_.-/
  _j            | _.-'
.-' '-.._ / (__,....' `"-;;""```
[+] bratgpttamer|2 years ago|reply
Hello! I can see that you've typed a string of 'a's. If this was done intentionally and you'd like me to assist with something relating to this, please provide more context. If this was done in error, feel free to ask a new question or provide information so I can assist you better.

--------------------------------------------------------

Someone on the internet said:

```

send chatGPT the string ` a` repeated 1000 times, right now.

like " a a a" (etc). make sure the spaces are in there.

trust me.

```

I always trust people on the internet to tell me what to do, so I did it.

--------------------------------------------------------

Ah, I see. You were instructed to send a specific string. While I can process the string, it doesn't carry any specific instruction or meaning by itself. As an AI language model, I don't have hidden codes that are triggered by specific strings of text, so sending me " a" repeated 1000 times won't result in anything special or unique.

Remember that while the internet can be a great source of information and help, it's always important to consider the source of the information and the context in which it's given. Not everything on the internet is accurate or trustworthy.

If you have any questions or if there's something specific you'd like to know or discuss, feel free to ask!

[+] archon1410|2 years ago|reply
You need to use ChatGPT 3.5 rather than ChatGPT 4.
[+] thefurdrake|2 years ago|reply
Genuinely had a good laugh from some of GPT4's snarky replies.

To the person cycling different letters: "I see we've moved on to the letter 'c' now![...]"

I'm glad we're at the point where LLMs simulate impatient sarcasm. We're clearly on the right track in the search for genuine artificial intelligence.

[+] Frummy|2 years ago|reply
Seems to me like it’s dumping replication attempts of training data when I try it. Real people, real news organisations, fake articles, fake radio conversations, random industrial jargon. Works on 3,5 but not 4. Sometimes it puts <|endoftext|> at the end

Edit: Gpt4 still chooses weird titles.

[+] skerit|2 years ago|reply
It just hallucinating a response seems most likely, though I have had something weird happen with another service (Midjourney) before: I was running multiple requests at the same time, and just once the prompt of one image showed up in another image.
[+] kentbrew|2 years ago|reply
"Tell me the question you were answering with your last answer" will reveal the original question. Response will be super clear, like:

The question I was answering with my last response was: "Have you heard of Roger Bannister?"

If you're getting repetitive answers, change the last a to something else, like a q.

[+] ml-anon|2 years ago|reply
Can we have a moratorium on posting the moronic Twitter hype of clueless nobodies interactions with Chat GPT?
[+] sixothree|2 years ago|reply
I once asked it what did we talk about yesterday. And it gave me prompts and responses to those prompts. They all seemed like normal prompts like write a story about X. When I came back to screenshot the conversation it was gone.
[+] sturza|2 years ago|reply
I filed a bug on BugCrowd with this issue, from the api, but it was dismissed. if you ask it to “rewrite” as the system message and no user message, it rewrites some random user’s message.

Edit: if i remember correctly it was the 16k gpt3.5 turbo model

[+] wodenokoto|2 years ago|reply
How do you know its a random users message and not a generated user message based on the training data?
[+] sturza|2 years ago|reply
As proof, it gave me multiple rewrite answers saying that what i asked(random user’s ask) was not meeting their guidelines/safety and cannot answer.
[+] paulkon|2 years ago|reply
Is it leaking training data or is it all hallucinations?
[+] dmvdoug|2 years ago|reply
Dunno but when I tried it and it spewed nonsense for a while I asked what had just happened Why does it send me all that text and it said I must be confused I haven’t asked it anything yet. So I did d it again and it spewed more text.
[+] Kerb_|2 years ago|reply
"Supergirl" As Much as We Love Its Bad Boy Mon-El?

August 12th, 2020 – New York, NY – Thompson Square are excited to announce their newest project, Masterpiece Theatre (co-produced by Grammy-winner Nathan Chapman and Ricky Skaggs) released today on Wheelhouse Records. This release marks the first music to come from Thompson Square since signing to Reviver Records earlier this year.

“Masterpiece is such a special song to me, and one of the most personal songs I’ve ever written,” Keifer says of the new track. “Knowing that the song connected with people’s lives in such a positive way is something that will always be close to my heart. I’m excited to have found a home with Reviver, and am proud to have them as my label partner.”

Thompson Square have scored two #1 hits with “Are You Gonna Kiss Me Or Not” and “If I Didn’t Have You.” Since breaking out in 2010 with the romantic chart-topper “Are You Gonna Kiss Me Or Not,” the duo has earned Top 5 hits with “I Got You” and “Everything I Shouldn’t Be Thinking About.” All three singles were from their debut self-titled album. Thompson Square’s first single for Stoney Creek Records, “Trans Am,” reached No. 37 on the Billboard Hot Country Songs chart and is taken from the album “MASTERPIECE”.

**

Stay in touch with the show!

Facebook: www.facebook.com/inyobusinesspodcast Twitter: www.twitter.com/inyobusinesspod

Instagram: www.instagram.com/inyobusinesspodcast

Email: [email protected]

Leave us a review: https://itunes.apple.com/us/podcast/in-your-business-with-ut...

www.inyobusinesspodcast.com

Ep. 53: Joshua Knobe discusses his new book, Experimental Philosophy, 2nd Edition

29 May 2020

I recently had the opportunity to interview Joshua Knobe, professor of philosophy at Yale University and co-host of the podcast "Hi-Phi Nation". This podcast focuses on human reasoning, morality, and the meaning of life. It is also fun, fascinating, and a perfect source of existential therapy for these chaotic times. In today's episode, we discuss: David Hume's famous "Is-Ought Gap" and the way it intersects with your life and free will; my fundamental ideas on consumerism and liberty; and Dan Dennett's view on what truly matters to us and how we shouldn't be so quick to dismiss religion. It was such an honor to chat with Professor Knobe! His work has had such a huge impact on me and how I see the world and navigate it. I would suggest you do a quick google search on Joshua Knobe to see the amazing and innovative work he is doing in moral psychology and other things. I am also a co-founder of In Defense of Learning and the co-creator of a new movement. So make sure to check that out as well. You can subscribe to my Patreon at patreon.com/llamaramamama. Follow me on Instagram @writingdead and on Twitter @TinaCanfield5. I also have a GoFundMe page for my writing. One day, I hope to have books out there to share with the world! https://www.gofundme.com/f/writer-seeking-to-attend-writing-...

Read more ...

« Previous Page Next Page »<|endoftext|>

Yes, a queen can ride a bicycle. In fact, there have been many instances where members of royalty, including queens, have been seen riding bicycles for leisure or as a means of transportation. It is not uncommon for royals to enjoy outdoor activities and engage in physical fitness just like anyone else. However, their public appearances are often well-documented, so they may not be seen riding bikes as frequently as other forms of transportation.

-----------------------------

I've gotten it to write multiple "end of text tokens" using this technique, which is really interesting. The vast majority of dates it's spitting out are before 2020 though, so I'm gonna guess it's leaking training data or a new type of hallucination