WingNews logo WingNews
top | item 37739297

(no title)

mecredis | 2 years ago

It's kind of wild that these tools just transfer a copy of these models every time they're spun up (whether it's to a Google Colab notebook or a local machine.)

This must mean Hugging Face's bandwidth bill must be crazy, or am I missing something (maybe they have a peering agreement? heavily caching things?)

discuss

order

satertek|2 years ago

Their Python module caches the downloads, which is checked before downloading them again...but you're probably not wrong on the crazy bandwidth bill. Looks like they have crazy VC money though, considering the current climate.

minimaxir|2 years ago

The Colab notebooks are a fresh and independent session with no caching.

civilitty|2 years ago

Unmetered 10+ gigabit connections were on the order of $1/mbit/mo wholesale over a decade ago when I priced out a custom CDN so for the cost of 100 TB of data transfer out of AWS you could get a 24/7 sustained 10gbit/s (>3 PB per month at 100% utilization).

Bandwidth has always been crazy cheap.

hotnfresh|2 years ago

Not all connections are created equal. Even some big providers clearly have iffy peering agreements upstream that’ll manifest as terrible performance if you have a widely-geographically-distributed bandwidth-heavy load.

colechristensen|2 years ago

Indeed. If you're not using a cloud provider bandwidth is extremely cheap.

In fact locally I can get a 10 gbps home internet unmetered connection for $300/mo.

I'm not sure how they'd react if I transferred 1 PB/mo though :)

morkalork|2 years ago

If you host copies of your data with a few big providers could you do something smart like detect and redirect requests from AWS to an S3 bucket and not pay for bandwidth leaving the provider?

anonylizard|2 years ago

Huggingface has a strategic partnership with AWS.

1. AWS is far behind Azure and GCP in AI, so they gotta partner up to gain credibility.

2. Huggingface probably does face insane bills compared to github. But AWS can probably develop some optimizations to save bandwidth costs. There's 100% some sort of generalized differential storage method being developed for AI models.

fomine3|2 years ago

AWS egress traffic charge is just outrageous so they can easily offer huge discount without improvement

jandrese|2 years ago

One doesn't usually opt for AWS when their goal is to reduce transfer costs.

toddmorey|2 years ago

Is hugging face just a model repository like GitHub is a code repository? Seems you can rent compute both cpu & gpu, but you are right that most models seem to be run elsewhere.

pdntspa|2 years ago

I really wish I could configure this crap to cache somewhere other than my C: drive

Or better yet, how about asking me where I want to store my models?

thulle|2 years ago

On linux there's the XDG_CACHE_HOME env variable for pip, but strangely enough there doesn't seem to be an windows equivalent.

callalex|2 years ago

I haven’t used windows in a while but I thought it supported some form of cross-volume symlink? Or at least mounting an image stored on another volume to an arbitrary path.