[artembugara]

thanks, this part is clear to me.

but I need to understand 20 x 1k token throughput

I assume it just might be too early to know the answer

[Tostino]

I legitimately cannot think of any hardware that will get you to that throughput over that many streams with any of the hardware I know of (I don't work in the server space so there may be some new stuff I am unaware of).

[artembugara]

oh, I totally understand that I'd need multiple GPUs. I'd just want to know what GPU specifically and how many