[megabless123]

noob question: why would increased demand result in decreased intelligence?

[exitb]

An operator at load capacity can either refuse requests, or move the knobs (quantization, thinking time) so requests process faster. Both of those things make customers unhappy, but only one is obvious.

[codeflo]

This is intentional? I think delivering lower quality than what was advertised and benchmarked is borderline fraud, but YMMV.

[sh3rl0ck]

I'd wager that lower tok/s vs lower quality of output would be two very different knobs to turn.

[awestroke]

I've seen some issues with garbage tokens (seemed to come from a completely different session, mentioned code I've never seen before, repeated lines over and over) during high load, suspect anthropic have some threading bugs or race conditions in their caching/inference code that only happen during very high load

[vidarh]

It would happen if they quietly decide to serve up more aggressively distilled / quantised / smaller models when under load.

[seunosewa]

Or just reducing the reasoning tokens.

[chrisjj]

They advertise the Opus 4.5 model. Secretly substituting a cheaper one to save costs would be fraud.

[Wheaties466]

from what I understand this can come from the batching of requests.

[chrisjj]

So, a known bug?