top | item 40996996 (no title) Bumblonono | 1 year ago It fits a 4090. Nvidia lists the models and therefore i assume 24gig is min discuss order hn newest michaelt|1 year ago A 4090 will just narrowly fit a 34B parameter model at 4-bit quantisation.A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.
michaelt|1 year ago A 4090 will just narrowly fit a 34B parameter model at 4-bit quantisation.A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.
michaelt|1 year ago
A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.