[vonseel]

I am no where near an expert on this subject, and this information is from a few months ago so maybe it's outdated, but people on Reddit[1] are claiming running the llama with 65B parameters would need like 20K+ of GPUs. A 40GB A100 looks like it's almost $8K on Amazon, and I'm sure you could do a lot with just one of those, but that's already beyond your $5K budget.

[1] https://www.reddit.com/r/MachineLearning/comments/11i4olx/d_...

I'll let others chime in but you could still probably build something really powerful within your budget that is able to run various AI tasks.

[logicchains]

You can get around 4-5 tokens per second on the 65B LLaMA with a 32 core 256GB ram Ryzen CPU, not sure how much it costs to build but can rent one from Hetzner for around two hundred bucks a month.