Noob question (I only learned how to use ollama a few days ago): what is the easiest way to run this DeepSeek-R1-Distill-Qwen-32B model that is not listed on ollama (or any other non-listed model) on my computer ?
If you are specifically running it for coding, I'm satisfied with using it via continue.dev in VS Code. You can download a bunch of models with ollama, configure them into continue, and then there is a drop-down to switch models. I find myself swapping to smaller models for syntax reminders, and larger models for beefier questions.
I only use it for chatting about the code - while this setup also lets the AI edit your code, I don't find the code good enough to risk it. I get more value from reading the thought process, evaluating it, and the cherry picking which bits of its code I really want.
In any case, if that sounds like the experience you want and you already run ollama, you would just need to install the continue.dev VS Code extension, and then go to its settings to configure which models you want in the drop-down.
Search for a GGUF on Hugging Face and look for a "use this model" menu, then click the Ollama option and it should give you something to copy and paste that looks like this:
ollama run hf.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF:IQ1_M
Whenever they have an alias like this, they usually (always?) have a model with the same checksum but a more descriptive name, e.g. the checksum 38056bbcbb2d corresponds with both of these:
I prefer to use the longer name, so I know which model I'm running. In this particular case, it's confusing that they grouped the qwen and llama fine tunes with R1, because they're not R1.
codingdave|1 year ago
I only use it for chatting about the code - while this setup also lets the AI edit your code, I don't find the code good enough to risk it. I get more value from reading the thought process, evaluating it, and the cherry picking which bits of its code I really want.
In any case, if that sounds like the experience you want and you already run ollama, you would just need to install the continue.dev VS Code extension, and then go to its settings to configure which models you want in the drop-down.
rahimnathwani|1 year ago
mike31fr|1 year ago
simonw|1 year ago
mike31fr|1 year ago
nyrikki|1 year ago
https://ollama.com/library/deepseek-r1
rahimnathwani|1 year ago
https://ollama.com/library/deepseek-r1:32b
https://ollama.com/library/deepseek-r1:32b-qwen-distill-q4_K...
I prefer to use the longer name, so I know which model I'm running. In this particular case, it's confusing that they grouped the qwen and llama fine tunes with R1, because they're not R1.
marpstar|1 year ago