Show HN: a desktop app to discover, download, and run offline LLMs

[pearlsteinj]

Wow, this makes running LLMs locally significantly more approachable. Very cool!

[leeholim]

Agreed!

[asselinpaul]

This is great! CPU only for now or can it leverage GPU for sped up inference?

[ybu]

Focus is CPU for now - but it'd be very useful to have GPU support. As a baseline, this can support anything that llama.cpp. Relevant link: https://github.com/ggerganov/llama.cpp#blas-build

[dtagames]

Looks very useful! Can't wait to try the Windows version. Thanks!

[ybu]

Follow up: Windows release is out.

[ybu]

hopefully soon!