[sounds]

Have you ever tried to run a model from huggingface on an AMD GPU?

Semiconductor fabrication is a high risk business.

Nvidia invested heavily in CUDA and out-competed AMD (and Intel). They are working hard to keep their edge in developer mindshare, while chasing hardware profits at the same time.

[phkahler]

>> Have you ever tried to run a model from huggingface on an AMD GPU?

Yes. I'd never touched any of that stuff and then one day decided to give it a shot. Some how-to told me how to run something on Linux which had a choice of a few different LLMs. I picked one of the small ones (under 10B) and had it running on my AMD APU inside of 15 minutes. The weights were IIRC downloaded from huggingface. The wrapper was not. Anyway, what's the problem?

BTW that convinced me that small LLMs are basically worthless. IMA need to go bigger next time. BTW my "old" 5700G has 64GB of RAM, next build I'll go at least double that.

[amelius]

> Have you ever tried to run a model from huggingface on an AMD GPU?

No, but seeing how easily they run on Apple hardware, I don't understand your point, to be honest.