[mattwad]

i run a home media server, can't wait to be able to add my own LLM service. it's just a matter of time for it to be something i can install over the weekend with proper hardware

[proaralyst]

Have you tried https://ollama.com/ ? You may find you already can

[gryfft]

  git clone https://github.com/ggerganov/llama.cpp
  cd llama.cpp
  make
  ./server -m models/7B/ggml-model.gguf -c 2048

I don't think it'll take you the whole weekend :)