top | item 44378499 Give Me FP32 or Give Me Death? 3 points| spindump8930 | 8 months ago |arxiv.org 2 comments order hn newest spindump8930|8 months ago People often don't understand why LLMs can be non deterministic even with deterministic seeding, temperature, sampling. This paper shows how bad it can be with different hardware and gpu hosts. incomingpain|8 months ago Q4 is plenty for me, I dont have the budget for FP32 lol.If money wasnt a thing, id probably not be going above Q8.
spindump8930|8 months ago People often don't understand why LLMs can be non deterministic even with deterministic seeding, temperature, sampling. This paper shows how bad it can be with different hardware and gpu hosts.
incomingpain|8 months ago Q4 is plenty for me, I dont have the budget for FP32 lol.If money wasnt a thing, id probably not be going above Q8.
spindump8930|8 months ago
incomingpain|8 months ago
If money wasnt a thing, id probably not be going above Q8.