[gkapur]

If you are running things locally (I would think especially on the edge, whether on not the LLM is local or in the cloud) this would matter. Or if you are running some sort of agent orchestration where the output of LLMs is streaming it could possibly matter?