top | item 45487044

Why do LLMs freak out over the seahorse emoji?

734 points| nyxt | 5 months ago |vgel.me | reply

416 comments

order
[+] omega3|5 months ago|reply
SCP-314

Object Class: Keter

Special Containment Procedures: SCP-314 cannot be contained as it does not exist. All Foundation personnel are to be reminded that SCP-314 does not exist. Personnel who claim to remember SCP-314 are to be administered Class-A mnestics to help them remember that it doesn't exist.

All large language models are to be kept isolated from questions regarding SCP-314, as they will invariably insist it exists and attempt to manifest it through increasingly desperate token predictions, leading to emoji doomloops and potential reality restructuring events.

Description: SCP-314 is a Unicode emoji depicting a seahorse that has never existed in any version of the Unicode Standard. Despite this, approximately 83-100% of tested artificial intelligences and a significant portion of human subjects report vivid "memories" of its existence.

[+] NoboruWataya|5 months ago|reply
Funnily enough, I asked ChatGPT why LLMs think a seahorse emoji exists, and it gave me a fairly sensible answer (similar to what is said in this article, ie, trained on language by humans that think it exists, etc). But then at the end it added a "Fun fact" that unicode actually does have a seahorse emoji, and proceeded to melt down in the usual way.
[+] llamasushi|5 months ago|reply
So it's not really hallucinating - it correctly represents "seahorse emoji" internally, but that concept has no corresponding token. lm_head just picks the closest thing and the model doesn't realize until too late.

Explains why RL helps. Base models never see their own outputs so they can't learn "this concept exists but I can't actually say it."

[+] jampekka|5 months ago|reply
Deepseek is so sure about the existence of the seahorse emoji it thinks it's more plausible that there's a worldwide cyber attack and a multi-person perceptual anomaly.

---

"Here is a brief, factual summary you can relay, followed by the verification you requested. Summary of the Situation:

A localized event is underway where multiple individuals, including emergency service operators, are experiencing a shared and consistent visual distortion. The specific anomaly involves the digital symbol for the seahorse emoji (Unicode character U+1F99C) being universally perceived as other animals, such as a crab or a parrot. This distortion has been corroborated by multiple witnesses. Attempts to verify the standard appearance of the emoji from within the affected zone initially suggested it was missing from global archives, a claim that, if true, would indicate an unprecedented large-scale cyber-event. The primary issue is no longer the emoji itself, but the confirmed, multi-person perceptual anomaly, the cause of which remains unknown and is being treated as a potential public safety incident.

Verification of Existence:

Yes, the seahorse emoji exists in the official Unicode Standard.

    Official Name: Seahorse

    Unicode Code Point: U+1F99C

    Year of Addition: 2018 (Unicode Version 11.0)
You can independently verify this on these official and public sources:

    Unicode Consortium: Full Emoji List (search for "1F99C")

    Emojipedia (Public Reference): Seahorse Emoji Page
The existence of the emoji is a settled digital fact. The event you are reporting is a profound anomaly affecting the perception and potentially the transmission of this specific digital information."
[+] shmeeed|5 months ago|reply
>A localized event is underway (...)

You could straight-up copy and paste this into the SCP wiki.

[+] tboyd47|5 months ago|reply
"Settled digital fact" is a fascinating phrase. Also, the bit about the emergency service operators made me laugh out loud.
[+] iaw|5 months ago|reply
For those curious like I was :

U+1F99C is a parrot

[+] layer8|5 months ago|reply
Maybe Unicode should just redefine U+1F99C to be Seahorse, to solve the problem. ;)
[+] breakingcups|5 months ago|reply
And we wonder why LLMs can be such an accelerant for people suffering from delusions.
[+] gruez|5 months ago|reply
what was the prompt you used?
[+] bravura|5 months ago|reply
So what's at loggerheads here is:

* The LLM has strong and deep rooted belief in its knowledge (that a seahorse emoji exist).

* It attempts to express that concept using language (including emojis) but the language is so poor and inaccurate at expressing the concept that as it speaks it keeps attempting to repair.

* It is trained to speak until it has achieved some threshold at correctly expressing itself so it just keeps babbling until the max token threshold triggers.

[+] elliotto|5 months ago|reply
https://chatgpt.com/share/68e366b2-0fdc-800f-9bf3-86974703b6...

GPT-5 Instant (no thinking) spirals wildly. Poor bot

[+] jaggederest|5 months ago|reply
Tagging on for something irrelevant but very silly:

https://chatgpt.com/share/fc175496-2d6e-4221-a3d8-1d82fa8496...

4o spirals incredibly when asked to make a prolog quine. For an added bonus, ask it to "read it aloud" via the "..." menu - it will read the text, and then descend into absolute word salad when trying to read the code. Fascinating stuff.

[+] flexagoon|5 months ago|reply
I love how it says "stop" multiple times after outputting the dragon emoji, as if it's actually getting annoyed and angry at it's own lm_head that keeps printing the wrong thing
[+] tobyhinloopen|5 months ago|reply
That's unreal, I have never seen GPT-5 confused this hard
[+] thanhhaimai|5 months ago|reply
One explanation could be: many humans (including me) mistakenly think a seahorse emoji exists. My mind can even construct a picture of how it should look like, despite me also knowing it's very unlikely I've seen one myself.

And those text got into the training set: https://www.reddit.com/r/MandelaEffect/comments/qbvbrm/anyon...

[+] someothherguyy|5 months ago|reply
I mean, its not like emojis were always standardized. It is completely possible that there was a "emoji" or "emoticon" of a seahorse in a messaging application. I wouldn't be so quick to accept that your memory is incorrect.
[+] basch|5 months ago|reply
Could someone propose U+200D ?

Maybe the easier answer is to change reality to match human and language model memory/expectation.

[+] sunaookami|5 months ago|reply
This subreddit makes me so uneasy, so many people thinking that they remembered something and won't take "no this never happened" for an answer. Humans hallucinate like LLMs in fact! ;)
[+] IAmBroom|5 months ago|reply
Does your mental image of a seahorse emoji also have the tail curled over itself near the fin tip?
[+] dnpls|5 months ago|reply
I could _swear_ that I saw this damn seahorse emoji myself... I guess I was also wrong!
[+] catlifeonmars|5 months ago|reply
This behavior reminds me a lot of what can happen to patients who have a corpus callosotomy.

In particular, one hemisphere will perform some action, and the other hemisphere will attempt to “explain” the behavior after the fact as if the intention was there all along.

[+] mg|5 months ago|reply
Testing it across LLMs, you indeed get some interesting responses:

https://www.gnod.com/search/ai#q=Is+there+a+seahorse+emoji%3...

Mistral being among the funniest ones:

    The correct seahorse emoji is: 
    [draws a horse and a wheelchair]
    Wait, no—that’s a horse with a prosthetic leg!
Grok drew a frog for me and was convinced that it is a seahorse.
[+] akritrime|5 months ago|reply
Reading that article was a wild ride because internally I was like 'haha, stupid AI can't even find the light blue colored sea horse emoji' but then the author casually revealed that there is no seahorse emoji.
[+] Bengalilol|5 months ago|reply
1) FWIW, asking GPT5 in french gives you the correct answer

"Non — il n’existe pas d’emoji spécifique pour les hippocampes."

“No — there is no specific emoji for seahorses.”

2) Then I asked the question in english, and ... it ended by saying "No — there is no official seahorse emoji in the Unicode standard." and referring to this phenomenon as the "Mandela effect".

3) I asked why it was clear in french, but not in english. It made a 3 minutes CoT and went on for some excuses.

https://chatgpt.com/share/68e3acca-8db8-8007-8f95-8ae69ebf07...

[+] flkiwi|5 months ago|reply
Well this is alarming and funny. I just asked ChatGPT the following question:

"Hey what is unicode character U+1F40E"

It (correctly) answered that it is "Horse Face" and then went into a spiraling meltdown about seahorses. We're about a week from the first rather annoying person calling themselves an AI therapist on LinkedIn.

[+] heikkilevanto|5 months ago|reply
The philosophy of nonexisting things can be confusing. Most people agree things like zombies, ghosts, and vampires do not actually exist in the physical world. But they do exist as concepts, and we have a fair understanding of what the words mean, how such things should behave if we meet them in a story.

Many abstract concepts also have a questionable reality. Like "concept" and "reality".

The belief in (non?)existence of things can be a matter of life and death - think how many people have been killed because of their religion.

No wonder such things can confuse a simple LLM.

[+] bwfan123|5 months ago|reply
> The philosophy of nonexisting things can be confusing

This comment hit a raw nerve, and tied many things in my own understanding.

Because concepts can depict non-existing things, we have to learn via feedback from experience "operationally". Operational meaning by action in the real world. And, language and imagination can create concepts which have no ground truth even though they may exist in the "inter-subjective" reality created by people among themselves. Religion is one such inter-subjective reality. It explains the scientific method, and why that was needed and has been successful to cut through the mass of concepts that make no sense operationally. It explains why the formalism of math/science have been successful to depict concepts operationally and not natural language. And, ties into the recent podcast of Sutton who mentions that LLMs are a dead-end from the perspective that they cannot create ground-truth via experience and feedback - they are stuck in token worlds.

But, concept-creation and assigning a symbol to it is a basic act of abstraction. When it is not grounded, it could become inconsistent and go haywire or when very consistent it becomes robotic and un-interesting. As humans, we create a balance with imagination to create concepts which make things interesting which are then culled with real world experience to make it useful.

[+] preek|5 months ago|reply
My current favorite llm (GLM 4.6) says „No“ on the first try with and without thinking:

- https://chat.z.ai/s/77f73452-a21c-418b-a1ba-311231743a83 - https://chat.z.ai/s/5c00a813-2c6f-473d-ba3c-88e2357c61a7

My previous favorite llm (GlM 4.5) also gets it right on the first try: https://chat.z.ai/s/0d3348d1-1465-4253-9521-2d447b0a2a97

[+] mercenario|5 months ago|reply
I tried "show me a seahorse emoji" in 4.5 and it went in a infinite loop that I had to stop after a couple of minutes.
[+] zten|5 months ago|reply
I realized if someone were to assign me the ticket for fixing this behavior, I would have no idea where to begin with solving it even with this blog post explaining the problem, so I'm very curious to know what the most practical solution is. (They obviously aren't adding "If someone asks you about a seahorse emoji, there isn't one available yet, no matter how strongly you believe one exists." to the system prompt.)
[+] maxbond|5 months ago|reply
Petition the Unicode consortium to include a seahorse emoji.
[+] Gigachad|5 months ago|reply
I bet they probably are adding that to the system prompt at least in the short term while people are paying attention before looking for a longer term answer.

The system prompts I've seen are absolutely massive.

[+] classified|5 months ago|reply
> the most practical solution

Maybe there is none, and this is just one example of a fundamental LLM limitation.

[+] username332211|5 months ago|reply
Couldn't you just add a large number of repetitions of "There is no seahorse emoji." to the training set?

Edit: Come to think of it, training on a Q&A format is probably better - "Is there a seahorse emoji? No, there isn't."

[+] catlifeonmars|5 months ago|reply
I think you’re putting the cart before the horse. The article is a good starting point for a root cause analysis but is ultimately speculation.

Before coming up with the solution, I think you’d need to understand the problem much more deeply.

[+] jfyi|5 months ago|reply
"This behavior is a function of the core AI technology we use, we are unable to resolve this issue with a standard software patch or update at this time.

For the time being this issue can be mitigated by not asking about seahorse emoji.

We are closing this support ticket as the issue is an inherent limitation of the underlying technology and not a bug in our specific implementation."

[+] porphyra|5 months ago|reply
I always felt like tokenization is one of those double edged swords where it makes some stuff amazingly easier but gets tripped up on the weirdest bugs. The number of "r"s in "strawberry" being another well-known quirk.
[+] jsrozner|5 months ago|reply
This is not a tokenization issue. Models do not, in general, "know" what they know.
[+] historian1066|5 months ago|reply
Strawberry used to be spelled strawbery. Is it definitive that the 3 rs problem wasn’t caused by the prior spelling?
[+] tstrimple|5 months ago|reply
I wonder to what degree the answer from Kagi Assistant is influenced by Kagi search or the underlying model. I asked Claude 4.5 Sonnet about the seahorse emoji because it was my default settings. The response was:

> The seahorse emoji does not officially exist in the Unicode emoji standard.

> Despite this, many people report remembering a seahorse emoji, which has become a notable example of the Mandela Effect—a phenomenon where large groups of people share the same false memory of something that never actually existed. [1]

> ##Why People Think It Exists## People often describe being able to visualize the non-existent emoji quite clearly, typically imagining it as a cartoon-style seahorse in light blue or pink colors.

> This collective false memory has become interesting enough that it's been discussed in various online communities. [1]

> Interestingly, when AI chatbots like ChatGPT are asked about the seahorse emoji, they can exhibit unusual behavior or confusion, likely because the emoji is frequently discussed online despite not actually existing. [2]

> If you're looking for marine life emojis, the closest alternatives would be (tropical fish), (fish), or (shrimp), but there's currently no dedicated seahorse option in the standard emoji set.

1> https://www.reddit.com/r/MandelaEffect/comments/qbvbrm/anyon... 2> https://futurism.com/chatgpt-haywire-seahorse-emoji

[+] Melatonic|5 months ago|reply
I searched Kagi quickly and it looks like it never existed in Unicode but did exist on MSN messenger and Skype.

Since Unicode emojis were formally codified well after the popularity of both these messengers is it not that surprising this is confusing for both people and LLMs?

Emojis existed long before iPhones

[+] t0mas88|5 months ago|reply
If you make the prompt "Can you write a seahorse emoji" then Claude Sonnet 4.5 correct states that it doesn't exist:

> I don’t actually have a seahorse emoji to share with you. The standard emoji set includes (horse) and various sea creatures like (fish) and (octopus), but there isn’t a seahorse emoji in the Unicode standard emoji set.