[jascha_eng]

idk I haven't really hit the point with any llm that it comes up with useful abstractions on its own unless those abstractions have been in the training data.

E.g. imagine building a google docs clone where you have different formatting options. Claude would happily build bold and italic for you but if afterwards you add headings, tables, colors, font size, etc. It would just produce a huge if/else tree instead of building a somewhat sensible text formatting abstraction.

Tbf I wouldn't actually know how to build this myself but e.g. bold and italic work together but if you add a "code block" thing that should probably not work with font color and putting a table inside that also makes no sense.

Claude might get some of these interactions intuitively correct but at some point you'll have so many NxM interactions between features that it just forgets half of them and then the experience becomes sloppy and crashes on all edge cases.

The point of good software engineering is to simplify the matrix to something that you can keep arguing about e.g. classify formatting options into categories and then you only have to argue and think about how those categories interact.

This is the kind of thing LLMs just aren't really good at if the problem space isn't in the training data already => doing anything remotely novel. And I haven't seen it improve at this either over the releases.

Maybe this kind of engineering will eventually be dead because claude can just brute force the infinitely growing if/else tree and keep it all in context but that does not seem very likely to me. So far we still have to think of these abstraction levels ourselves and then for the sub-problems I can apply agentic coding again.

Just need to make sure that Claude doesn't breach these abstractions, which it also happily does to take short cuts btw.

[nerdsniper]

FWIW I’ve used LLMs to invent new things. Not super groundbreaking fundamental research, but they were able to use physics to design a device that didn’t exist yet, from first principles.

[mrguyorama]

Pics or it didn't happen

More seriously, what in the world "novel" physics device did you invent?

[_345]

Would you share a bit more?