top | item 45621993 (no title) edude03 | 4 months ago $50k would be the cost to run it un-quantized, 10k could get you for example 4 5090 system, that would run the 671b q4 model which is 90% as good, which was the OPs target discuss order hn newest tkz1312|4 months ago which 671b quants can fit into 96GB VRAM? Everything I’m aware of needs hundreds at least (e.g. https://apxml.com/models/deepseek-r1-671b). lossolo|4 months ago 5090 is 32 GB so it's 128GB, not 96. load replies (1)
tkz1312|4 months ago which 671b quants can fit into 96GB VRAM? Everything I’m aware of needs hundreds at least (e.g. https://apxml.com/models/deepseek-r1-671b). lossolo|4 months ago 5090 is 32 GB so it's 128GB, not 96. load replies (1)
tkz1312|4 months ago
lossolo|4 months ago