[pyrale]

With AI, it feels like deterministic outcomes are not valued as experience taught us it should.

The absence of means to measure outcomes of these prompt documents makes me feel like the profession is regressing further into cargo culting.

[bandrami]

It's particularly puzzling because until a few months ago the unmistakable consensus at the fuzzy borderlands between development and operations was:

1. Reproducibility

2. Chain of custody/SBOM

3. Verification of artifacts of CI

All three of which are not simply difficult but in fact by nature impossible when using an LLM

[EdNutting]

That might be because AI is being pushed largely by leaders that do not have the experience you’re referring to. Determinism is grossly undervalued - look at how low the knowledge of formal verification is, let alone its deployment in the real world!

[XenophileJKO]

That is because nothing in the world is deterministic, they are just all varying degrees of probability.

[pyrale]

This rings hollow to me.

When my code compiles in the evening, it also compiles the next morning. When my code stops compiling, usually I can track the issue in the way my build changed.

Sure, my laptop may die while I'm working and so the second compilation may not end because of that, but that's not really comparable to a LLM giving me three different answers when given the same prompt three times. Saying that nothing is deterministic buries the distinction between these two behaviours.

Deterministic tools is something the developper community has worked very hard for in the past, and it's sad to see a new tool giving none of it.

[latexr]

That is called a deepity: a statement which sounds profound but is ultimately trivial and meaningless.

https://rationalwiki.org/wiki/Deepity

Determinism concerns itself with the predictability of the future from past and present states. If nothing were deterministic, you wouldn’t be able to set your clock or plan when to sow and when to harvest. You wouldn’t be able to drive a car or rest a glass on a table. You wouldn’t be able to type the exact same code today and tomorrow and trust it to compile identically. The only reason you can debug code is determinism, it is because you can make a prediction of what should happen and by inspecting what did happen you can can deduce what went wrong several steps before.