[coltonv]

Yes but if I set it above ~16K on my 32gb laptop it just OOMs. Am I doing something wrong?

[mekpro]

try enable flash attention and offload all layer to GPU