[XenophileJKO]

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplication. That could be eleminated if we wanted to buy restricting the ordering.

[bandrami]

True but at that point you're not really using "an LLM" in the usual sense, you're using a very resource-hungry database.