[k2so]

LLMs = Latency? That's how most of us perceive it. When examining the timing breakdown of a request on Claude, you'll notice that the majority of the time is spent in Content Download—essentially, decoding output tokens.

In the blog, I discuss how partial json validation can help in workflow driven LLM products.

Would love feedback on how I can improve, thanks!