Show HN: I made a VRAM Calculator in Hugging Face

[PieterBecking]

It's a chrome extension that automatically loads the specs from the Hugging Face model card into the calculation.

> To test it, install the extension (no registration/key needed) and navigate to a HF model page. Then click the "VRAM" icon on the top right to open the sidepanel.

You can specify quantization, batch size, sequence length, etc.

Works for inference & fine-tuning.

If it does not fit on the specified GPUs, it gives you an advise on how to still run it (e.g. lowering precision).

It is inspired at my work, where we were constantly exporting metrics from HF to estimate required hardware. Now, it saves us in the dev team quite some time and clients can use it, too.

Let me know what you think.

[reach-vb]

Nice! that's very cool!

[PieterBecking]

Thanks!

[rokizero]

i'm honestly surprised HF doesn't have this feature yet, very useful! will you publish the code on github?

any plans on adding more consumer-grade gpus?

[PieterBecking]

Hey I published the code here: https://github.com/NEBUL-AI/HF-VRAM-Extension

I've added the 4090 and 5090 as well now, make sure to get version 0.5 of the extension

[clemnt]

pretty cool!

[PieterBecking]

thx!