[lsy]

I think the analogy to high level programming languages misunderstands the value of abstraction and notation. You can’t reason about the behavior of an English prompt because English is underspecified. The value of code is that it has a fairly strong semantic correlation to machine operations, and reasoning about high level code is equivalent to reasoning about machine code. That’s why even with all this advancement we continue to check in code to our repositories and leave the sloppy English in our chat history.

[skydhash]

Yep. Any statement in python or others can be mapped to something that the machine will do. And it will be the same thing every single time (concurrency and race issue aside). There’s no english sentence that can be as clear.

We’ve created formal notation to shorten writing. And computation is formal notation that is actually useful. Why write pages of specs when I could write a few lines of code?

[WorldMaker]

There's also creative space inside the formal notation. It's not just "these are the known abstractions, please lego them together", the formal syntax and notation is just one part of the whole. The syntax and notation define the forms of poetry (here's the meter, here's the rhyme scheme, here's how the whitespace works), but as software developers we're still filling in the words that fit that meter and rhyme scheme and whitespace. We're adding the flowery metaphors in the way we choose variable names and the comments we choose to add and the order we define things or choose to use them.

Software developers can use the exact same "lego block" abstractions ("this code just multiplies two numbers") and tell very different stories with it ("this code is the formula for force power", "this code computes a probability of two events occurring", "this code gives us our progress bar state as the combination of two sub-processes", etc).

LLMs have only so many "stories" they are trained on, and so many ways of thinking about the "why" of a piece of code rather than mechanical "what".