[christina97]

Author is clearly confused about the Anthropic case. The request rate at these generation endpoints is so high that the current batching delay is effectively negligible.