[frognumber]

Missing data: I don't make a codex PR if it's nonsense.

Poor data: If I make one, I either if I want to:

a) Merge it (success)

b) Modify it (sometimes success, sometimes not). In one case, Codex made the wrong changes in all the right places, but it was still easier to work from that by hand.

c) Pick ideas from it (partial success)

So simple merge rates don't say much.

[osigurdson]

It isn't so much "poor" data as it is a fairly high bar for value generation. If it gets merged it is a fairly clear indicator that some value is created. If it doesn't get merged then it may be adding some value or it may not.

[frognumber]

There's a numerator and a denominator. The numerator is fine for what you're saying -- the number of merged PRs.

The denominator varies wildly based on whether or not the PR is made. If codex makes nonsense, I don't ask it to make a PR.