[nfg]

Interesting - these head to head comparisons you’re doing with the same model - what harnesses are you comparing, say Claude code / codex versus copilot cli?

> I'm not sure if its understood how bad it really is within the org.

I can’t speak to that, but there’s a lively culture of people using internal tooling who also extensively use 3p products on projects outside work and are in a reasonable position to assess how well GH copilot works.

[kasey_junk]

Yeah, I’m only interested in cli and non-interactive agent usage. I don’t compare say the vs code plugins, but do regularly compare say GitHub code reviews.

Those comparisons for instance have made us turn _off_ copilot pull requests entirely. All of the agents have false positives (as do humans) but copilot was having negative value in that context.