[afro88]

But gcc is part of it's training data so of course it spit out an autocomplete of a working compiler

/s

This is actually a nice case study in why agentic LLMs do kind of think. It's by no means the same code or compiler. It had to figure out lots and lots of problems along the way to get to the point of tests passing.

[the_fall]

> But gcc is part of it's training data so of course it spit out an autocomplete of a working compiler /s

Why the sarcasm tag? It is almost certainly trained on several compiler codebases, plus probably dozens of small "toy" C compilers created as hobby / school projects.

It's an interesting benchmark not because the LLM did something novel, but because it evidently stayed focused and maintained consistency long enough for a project of this complexity.