[petespeed]

I tried to change config.py to use ggml setting, but I did not see any request going to local llama-cpp.server. It keeps on asking for openai key. The local llama-cpp.server is up and I was able to make swagger calls for completion.

Assuming that may be I missed something, I deleted ~/.continue and trying to start from scratch (vs code message: Starting Continue Server ...). I do see FastAPI is up on http://localhost:65432/ still vscode dialog shows same message. Where I can see the logs what its trying to do? What am I missing?

Anyone else able to use local ggml?

[sestinj]

Hey there. I realized that the documentation is less clear than it should be on this, so I've made an update: https://continue.dev/docs/customization#local-models-with-gg...

The additional thing you need to do is set "default_model" to "ggml" in ~/.continue/config.py.