[just6979]

It only takes a cursory knowledge of what LLMs really are to understand why recreating tutorials is easy, but making actual new stuff that is well engineered (takes way way more than "passes the test suite") is difficult.

Actual novel stuff is so far out on the long tail of iterations that it's a gamble: it might pop up in an early run, or might take 2000 prompts and $20,000 worth of tokens. And it's still not really engineered, it's 10,000 monkeys with typewriters copying random shakespeare snippets off the chalkboard. At some point you'll get all of Hamlet, but most of the time you'll get garbage, and sometimes you'll get Romeo & The Taming of The Tempest.