[isotypic]

Do you have some examples of ones you found beyond what a human could straightforwardly figure out? I tried a bunch and they all seemed reasonable, so I would be interested in seeing - I didn't try all 400, for obvious reasons, so I don't doubt there are difficult ones.

I think regardless one of the reasons people are interested in it is that is a fairly simple logic puzzle - given some examples, extrapolate a pattern, execute the pattern - that humans achieve high accuracy on (a study linked on the website has ~84% accuracy for humans, some more recent study seems to put it closer to 75%). Yet ML approaches have yet to reach that level, in contrast to other problems ML has been applied to.

Given there is a large prize pool for the challenge, I would imagine actually training a model in the way you describe would already have been tried and is more difficult that it seems.

[ec109685]

I realize I didn’t scroll to other examples for one I found very hard.

I guess the question is whether someone who solves this will have cracked AGI as a necessary precondition or like other Turing tests that have been solved, someone will find a technique that isn’t broadly applicable to general intelligence.