[gwd]

I feel like a lot of evaluations are pretty clearly evaluations. Not sure how to add the messiness and grit that a real benchmark could have.

That said, apparently Gemini's internal thought process reveals that it thinks loads of things were simulations when they aren't; it's 99% sure news stories about Trump from Dec 2025 are a detailed simulation:

https://www.reddit.com/r/GeminiAI/comments/1qhadce/gemini_is...

ETA: From the article that put me on this:

> I write nonfiction about recent events in AI in a newsletter. According to its CoT while editing, Gemini 3 disagrees about the whole "nonfiction" part:

>> It seems I must treat this as a purely fictional scenario with 2025 as the date. Given that, I'm now focused on editing the text for flow, clarity, and internal consistency.

https://www.lesswrong.com/posts/8uKQyjrAgCcWpfmcs/gemini-3-i...