That doesn't make a difference here. Even with a nonzero temperature, an LLM could still be deterministic as long as you have control of its random seed. As the article says:
"This gets to my core point. What changes with LLMs isn’t primarily nondeterminism, unpredictability, or hallucination. It’s that the programming interface is functionally underspecified by default."
Even if you turn the temperature down to 0, it's not deterministic. Floating points are messy. If there is even a tiny difference when it comes to the order of operations on the actual GPU that's running the billions of parallelized floating point operations over and over, it's very possible to end up with changing top probability logits.
More to the point: is randomness of representation or implementation an inherent issue if the desired semantics of a program are still obeyed?
This is not really a point about whether LLMs can currently be used as English compilers, but more questioning whether determinism of the final machine code output is a critical property of a build system.
No, for the reasons given in the sibling comments: you won't want to be locked into a single model for the rest of time and, even if you did, floating point execution order will still cause non-determinism.
kibwen|24 days ago
"This gets to my core point. What changes with LLMs isn’t primarily nondeterminism, unpredictability, or hallucination. It’s that the programming interface is functionally underspecified by default."
helloplanets|24 days ago
abm53|24 days ago
This is not really a point about whether LLMs can currently be used as English compilers, but more questioning whether determinism of the final machine code output is a critical property of a build system.
nickm12|21 days ago
MyHonestOpinon|23 days ago
unknown|23 days ago
[deleted]