[wrs]

“Programs must be written for people to read, and only incidentally for machines to execute." — Harold Abelson

The purpose of high level languages is to make the structure of the code and data structures more explicit so it better captures the “actual” program model, which is in the mind of the programmer. Structured programming, type systems, modules, etc. are there to provide solid abstractions in which to express that model.

None of that applies to giving an LLM a feature idea in English and letting it run. (Though all of it is helpful for keeping an LLM from going completely off the rails.)

[skydhash]

The code for the machine to execute is opcodes realized by electric signals. But it's not a good tool to solve problems even when directly translated to assembly. All the other languages were invented for better human affordance and can map to assembly (and opcodes) with logical rules.

The last part is what matters. There's no such clear rules in LLMs behavior. Yes you can get to behave roughly like a rule, but there's no clear cut demarcation between what's in and what's not.