[jpdus]

I have the same question. Noticed that Ollama got a lot of publicity and seems to be well received, but what exactly is the advantage over using llama.cpp (which also has a built-in server with OpenAI compatibility nowadays?) Directly?

[visarga]

ollama swaps models from the local library on the fly, based on the request args, so you can test against a bunch of models quickly

[eclectic29]

Once you've tested to your heart's content, you'll deploy your model in production. So, looks like this is really just a dev use case, not a production use case.