The sycophancy from Claude is incredibly jarring. I agree with Ethan Mollick that this could turn out to have more of a disastrous impact than AI hallucination.
It's even a blocker for some design patterns. Ie it's difficult to discuss options and choose the best one when the AI agrees with you no matter what. If you ask "But what about X" it is more likely to reverse course and agree with your new position entirely.
It's really frustrating. I've come to loathe the agreeable tone because every time i see it i remember the times where i've hit this pain point in design.
I absolutely hate this too. And the only way around it is to manipulate it into cheerfully pointing out all the problems with something in a similarly sycophantic way.
Not chatty. Unbiased. Avoid use of emoji. Rather than "Let me know if..." style continuations, list a set of prompts to explore further topics. Do not start out with short sentences or smalltalk that does not meaningfully advance the response.
I want an intelligent agent (or one that pretends to be) that answers the question rather than something that I chat with.
As an aside, I like the further prompt exploration approach.
One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now).
However, that's a me thing - something that I do (or avoid doing) with how I interact with an LLM. As noted with the stories of people following the advice of an LLM, it isn't something that is universal.
Thank you so much for sharing your customizations and conversations, it is really fascinating and generous!
In both of your conversations, there is only one depth of interaction. Is that typical for your conversations? Do you have examples where you iterate?
I think your meta-cognitive take on the model is excellent:
"One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now)."
The only thing I would add is that, as a reference librarian, it can surface template decision-making patterns.
But I think it's more like that cognitive trick where you assign outcomes to the sides of a coin, and you flip it, and see how you brain reacts — it's not because you're going to use the coin to make the decision, but you're going to use the coin to induce information from your brain using System 1.
I'm struck by how often Claude responds with "You're right! Now let me look at the file..." when it can't know whether I'm right until after it looks at the file in question.
They have introduced a beta 'Preferences' feature recently under Custom Instructions. I've had good results from this preference setting in GPT:
Answer concisely when appropriate, more
extensively when necessary. Avoid rhetorical
flourishes, bonhomie, and (above all) cliches.
Take a forward-thinking view. OK to be mildly
positive and encouraging but NEVER sycophantic
or cloying. Above all, NEVER use the phrase
"You're absolutely right."
I just copied it into Claude's preferences field, we'll see if it helps.
What am I missing? I am not seeing this particular example as sycophantic.
Claude is saying, something like user's assertion is improbable but if it was the case, user needs to show/ prove some of the things in this table.
First, I think various models have various degrees of sycophancy — and that there are a lot of stereotypes out there. Often, the sycophancy, is a "shit sandwich" — in my experience, the models I interact with do push back, even when polite.
But for the broader question: I see sycophancy as a double‑edged sword.
• On one side, the Dunning–Kruger effect shows that unwarranted praise can reinforce over‑confidence and bad decisions.
• On the other, chronic imposter syndrome is real—many people underrate their own work and stall out. A bit of positive affect from an LLM can nudge them past that block.
So the issue isn't "praise = bad" but dose and context.
Ideally the model would:
1. mirror the user's confidence level (low → encourage, high → challenge), and
2. surface arguments for and against rather than blanket approval.
That's why I prefer treating politeness/enthusiasm as a tunable parameter—just like temperature or verbosity—rather than something to abolish.
In general, these all-or-nothing, catastrophizing narratives in AI (like in most places) often hide very interesting questions.
unshavedyak|7 months ago
It's really frustrating. I've come to loathe the agreeable tone because every time i see it i remember the times where i've hit this pain point in design.
ghc|7 months ago
danielbln|7 months ago
shagie|7 months ago
As an aside, I like the further prompt exploration approach.
An example of this from the other day - https://chatgpt.com/share/68767972-91a8-8011-b4b3-72d6545cc5... and https://chatgpt.com/share/6877cbe9-907c-8011-91c2-baa7d06ab4...
One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now).
However, that's a me thing - something that I do (or avoid doing) with how I interact with an LLM. As noted with the stories of people following the advice of an LLM, it isn't something that is universal.
lumbroso|7 months ago
In both of your conversations, there is only one depth of interaction. Is that typical for your conversations? Do you have examples where you iterate?
I think your meta-cognitive take on the model is excellent:
"One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now)."
The only thing I would add is that, as a reference librarian, it can surface template decision-making patterns.
But I think it's more like that cognitive trick where you assign outcomes to the sides of a coin, and you flip it, and see how you brain reacts — it's not because you're going to use the coin to make the decision, but you're going to use the coin to induce information from your brain using System 1.
organsnyder|7 months ago
CamperBob2|7 months ago
itemize123|7 months ago
lumbroso|7 months ago
But for the broader question: I see sycophancy as a double‑edged sword.
• On one side, the Dunning–Kruger effect shows that unwarranted praise can reinforce over‑confidence and bad decisions.
• On the other, chronic imposter syndrome is real—many people underrate their own work and stall out. A bit of positive affect from an LLM can nudge them past that block.
So the issue isn't "praise = bad" but dose and context.
Ideally the model would:
1. mirror the user's confidence level (low → encourage, high → challenge), and
2. surface arguments for and against rather than blanket approval.
That's why I prefer treating politeness/enthusiasm as a tunable parameter—just like temperature or verbosity—rather than something to abolish.
In general, these all-or-nothing, catastrophizing narratives in AI (like in most places) often hide very interesting questions.