WingNews logo WingNews
top | item 38933788

(no title)

teilo | 2 years ago

No quantization (8_0). The full 48GB model. As for token count, I haven't tested it on more than 200 or so.

discuss

order

pilotneko|2 years ago

Isn’t 8_0 8-bit quantization?

teilo|2 years ago

Sorry. That was a major brain fart. Yes. 8-bit quantization, and using 49G of RAM.