[aappleby]

What are you running that model on?

[kir-gadjello]

I just use openrouter, it's free for now. But I would pay 30-100$ to use it 24/7.

[aappleby]

Ah, I thought you meant you were running it locally.

[Aerroon]

Have you tried Minimax M2.5? How did it compare?

[FuckButtons]

A 3 bit quant will run on a 128gb MacBook Pro, it works pretty well.

[nl]

A 3 bit quant is quite a lot weaker than the OpenRouter version the OP is using.