[avaer]

Or, we could just use deterministic seeds in our LLM calls and solve the problem at the root.

Obviously this won't work if your tools are not deterministic, but reproducible builds is a well-trodden discipline.

[mapontosevenths]

This is actually a feature that OpenAI offers via the API. It doesn't work the way you want it to though. It makes it less random, not deterministic and they even warn you of that in the docs.