(no title)
IHLayman | 10 months ago
If you are using a system-controlling Nix (nix-darwin, NixOS…), it’s as easy as `hardware.services.ollama.enable=true` with maybe adding `.acceleration=“cuda”` to force GPU usage or `host=“0.0.0.0”` to allow connections to Ollama that are not local to your system. In a home-manager situation it is even easier: just include `pkgs.ollama` in your `home.packages`, with an `.override{}` for the same options above. That should be it, really.
I will say that if you have a more complex NixOS setup that patches the kernel or can’t lean on cachix for some reason that using the ollama package takes a long time to compile. My setup at home runs on a 3950X Threadripper and when Ollama compiles it uses all the cores at 99% for about 16 minutes.
No comments yet.