[d3m0t3p]

You can batch only if you have distinct chat in parallel,

[mlyle]

> > if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each)