top | item 44849427

(no title)

oli5679 | 6 months ago

Here is gpt 5 thinking posting all 20 questions verbatim. Appreciate I might get better results one question at a time.

https://chatgpt.com/share/6897a21b-25c0-8011-a10a-85850870da...

Pretty interesting - some contamination, some better answers, and it failed to write a sentence with all 5-letter-words. I’d have expected it to pass this one!

Simple example: “Every night, dreams swirl swiftly.

discuss

order

simsla|6 months ago

The problem is that ChatGPT doesn't really know letters, it writes in wordpieces (BPE), which may be one or more letters.

For example, something like "running" might get tokenizef like "runn"+"ing", being only two tokens for ChatGPT.

It'll learn to infer some of these things over the course of training, but limited.

Same reason it's not great at math.

DoctorOetker|6 months ago

you must be a bot:

d r e a m s (6 letters)

s w i f t l y (7 letters)

mutated_quant|6 months ago

Did the same question with Claude Sonnet 4, and I got a sentence that passed.

Every quiet mouse crept under thick brown boxes.