[1ba9115454]

I can't imagine this setup will get more than 1 token per second.

I would love to see Deepseek running on premise with a decent TPS.

[thomquaid]

It says 4.25 TPS in the first para.

[ricardobeat]

Honest mistake. Some people think HN is just a series of short tweets and haven’t realized they are links yet!

[weatherlight]

That's still pretty slow, considering there's that "thinking" phase.

[october8140]

You can get 1t/s on a raspberry pi.

https://youtu.be/o1sN1lB76EA?si=i8ecEBjLdV0zewFQ

[klohto]

this has nothing to do with the full 671B and the ollama models are distilled qwen2.5