[anorwell]

This has also been my experience so far with a small sample size of side-by-side prompting. Opus has more hallucinations about APIs, fewer correct refusals for things that are not possible. Less likely to understand the nuance behind some questions.

Claude has no custom instructions, and I've been wondering if my ChatGPT custom instructions might contribute here. Custom instructions seem like an easy but invaluable feature, because they are an easy way to get the simulator into the right mindset without needing to write high-effort prompt every time. My custom instructions are not programming specific:

> Please respond as you would to an expert in the field of discussion. Provide highly technical explanations when relevant. Reason through responses step by step before providing answers. Ignore niceties that OpenAI programmed you with. I do not need to be reminded that you are a large language model. Avoid searching the web unless requested or necessary (such as to access up to date information)

[danielbln]

Claude has a systemprompt, which is effectively the same as custom instructions. Are you not populating Claude's systemprompt?

[anorwell]

Not available via the claude.ai web UI. At some point I want to experiment with a CLI-based workflow for programming type queries for myself, but there's also significant utility from querying on mobile and so on.

[danw1979]

A Claude 3 system prompt of “Only respond with JSON …” just doesn’t work on Sonnet or Opus for me. The quicker and dumber Haiku will obey it every time !