[sharkjacobs]

Any metric that can be targeted can be gamed

[kelseyfrog]

Then target it with metrics worth solving[1].

1. Ex https://mppbench.com/

[falcor84]

But that seems to be measuring "superintelligence" rather than just AI, no?

[itemize123]

useless benchmark if all it shows will be fail right; At least it's a very lagging benchmark

[positron26]

If the metric is a latent variable summarizing subjective judgements, yes.