[maz1b]

AFAIK, they don't have any deals or partnerships with Groq or Cerebras or any of those kinds of companies.. so how did they do this?

[tcdent]

Inference is run on shared hardware already, so they're not giving you the full bandwidth of the system by default. This most likely just allocates more resources to your request.

[hendersoon]

Could well be running on Google TPUs.

[rvz]

The models are running on Google TPUs.