[jampa]

I am using API mode, and it's clear that there are times when the Claude model just gives up. And it is very noticeable because the model just does the most dumb things possible.

"You have a bug in line 23." "Oh yes, this solution is bugged, let me delete the whole feature." That one-line fix I could make even with ChatGPT 3.5 can't just happen. Workflows that I use and are very reproducible start to flake and then fail.

After a certain number of tokens per day, it becomes unusable. I like Claude, but I don't understand why they would do this.

[arcanemachiner]

Robbing Peter to pay Paul. They are probably resource-constrained, and have determined that it's better to supply a worse answer to more people than to supply a good answer to some while refusing others. Especially knowing that most people probably don't need the best answer 100% of the time.

[chrisjj]

> Especially knowing that most people probably don't need the best answer 100% of the time.

More: probably don't know if they've got a good answer 100% of the time.

It is interesting to note that this trickery is workable only where the best answers are sufficiently poor. Imagine they ran almost any other kind of online service such email, stock prices or internet banking. Occasionally delivering only half the emails would trigger a customer exodus. But if normal service lost a quarter of emails, they'd have only customers who'd likely never notice half missing.

[arresin]

Right. You can launder quantization that way by muddying the waters of discourse about the model.

[DanielHall]

I encountered the same situation too; Claude has 'become lazy'.