[lsy]

At what point do LLMs enable bad engineering practices, if instead of working to abstract or encapsulate toilsome programming tasks we point an expensive slot machine at them and generate a bunch of verbose code and carry on? I'm not sure where the tradeoff leads if there's no longer a pain signal for things that need to be re-thought or re-architected. And when anyone does create a new framework or abstraction, it doesn't have enough prior art for an LLM to adeptly generate, and fails to gain traction.

[michaelrpeskin]

How much of "good engineering practices" exist because we're trying to make it easy for humans to work with the code?

Pick your favorite GoF design pattern. Is that they best way to do it for the computer or the best way to do it for the developer?

I'm just making this up now, maybe it's not the greatest example; but, let's consider the "visitor" pattern.

There's some framework that does a big loop and calls the visit() function on an object. If you want to add a new type, you inherit from that interface, put visit() on your function and all is well. From a "good" engineering practice, this makes sense to a developer, you don't have to touch much code and your stuff lives in it's own little area. That all feels right to us as developers because we don't have a big context window.

But what if your code was all generated code, and if you want to add a new type to do something that would have been done in visit(). You tell the LLM "add this new functionality to the loop for this type of object". Maybe it does a case statement and puts the stuff right in the loop. That "feels" bad if there's a human in the loop, but does it matter to the computer?

Yes, we're early LLMs aren't deterministic, and verification may be hard now. But that may change.

In the context of a higher-level language, y=x/3 and y=x/4 look the same, but I bet the generated assembly does a shift on the latter and a multiply-by-a-constant on the former. While the "developer interface", the source code, looks similar (like writing to a visitor pattern), the generated assembly will look different. Do we care?

[wrs]

LLMs have limited working memory, like humans, and most of the practices that increase human programming effectiveness increase LLM effectiveness too. In fact more so, because LLMs are goldfish that retain no mental model between runs, so the docs had better be good, abstractions tight, and coding practices consistent such that code makes sense locally and globally.

[skydhash]

Code is a design tool, just like lines on an engineering drawing. Most times you do not care if it was with a pen or a pencil, or if it was printed out. But you do care about the cross section of the thing depicted. The only time you care about whether it’s pen or pencil is for preservation.

So I don’t care about assembly because it does not matter usually in any metric. I design using code because that’s how I communicate intent.

If you learn how to draw, very quickly, you find that no one talks about lines (which is mostly all you do), you will hear about shapes, texture, edges, values, balance…. It’s in these higher abstractions intent resides.

Same with coding. No ones thinks in keywords, brackets, or lines of code. Instead, you quickly build higher abstractions and that’s where you live in. The pros is that those concepts habe no ambiguity.

[chrisweekly]

Great Q, and your framing "there's no longer a pain signal for things that need to be re-thought or re-architected" perfectly encapsulates a concern I hadn't yet articulated so cleanly. Thanks for that!