[_1]

You might need to play around with the default settings. One of the first models I tried running on my Mac was really slow.. Turns out it was preallocating a long context window that wouldn't fit in the GPU memory, so it ran on the CPU.

[psychoslave]

Can you recommend some tutorial?

[psychoslave]

Self response: https://github.com/nordeim/running_LLMs_locally