[lpzimm]

Not exactly the same as this study, but I'll ask questions to LLMs with and without subtle hints to see if it changes the answer and it almost always does. For example, paraphrased:

No hint: "I have an otherwise unused variable that I want to use to record things for the debugger, but I find it's often optimized out. How do I prevent this from happening?"

Answer: 1. Mark it as volatile (...)

Hint: "I have an otherwise unused variable that I want to use to record things for the debugger, but I find it's often optimized out. Can I solve this with the volatile keyword or is that a misconception?"

Answer: Using volatile is a common suggestion to prevent optimizations, but it does not guarantee that an unused variable will not be optimized out. Try (...)

This is Claude 3.7 Sonnet.

[pixl97]

I mean, this sounds along the lines of human conversations that go like

P1 "Hey, I'm doing A but X is happening"

P2 "Have you tried doing Y?

P1 "Actually, yea I am doing A.Y and X is still occurring"

P2 "Oh, you have the special case where you need to do A.Z"

What happens when you ask your first question with something like "what is the best practice to prevent this from happening"

[lpzimm]

Oh sorry, these are two separate chats, I wasn't clear. I would agree that if I had asked them in the same chat it would sound pretty normal.

When I ask about best practices it does still give me the volatile keyword. (I don't even think that's wrong, when I threw it in Godbolt with -O3 or -Os I couldn't find a compiler that optimized it away.)