Pretty sure that's cos every AI company wants to have a copy of all the worlds information stored in each of their data centers, and hard drives are the best storage medium for that.
Design your training strategy carefully and you can do streaming rather than random reads from the drives and get enough performance.
To be honest, they mostly have 8 TB and up in stock at the moment, but the price was around 35 EUR for the 1 TB drive which is kind of insane when similar drives at local stores cost 110 EUR right now - I remember when those Seagate Barracuda drives were about 40-50 EUR brand new!
I still use SSDs for boot drives, video games and also fragmented data (like Garage for something S3 compatible, which chunks the files), but for backups and longer term storage HDDs are great.
dns_snek|15 days ago
londons_explore|15 days ago
Design your training strategy carefully and you can do streaming rather than random reads from the drives and get enough performance.
KronisLV|14 days ago
Recently got some 1 TB ones from Datablocks as backups, did a short writeup on my blog: https://blog.kronis.dev/blog/buying-some-drives-from-datablo... (though a better post made its way on HN by someone else a while ago, too)
To be honest, they mostly have 8 TB and up in stock at the moment, but the price was around 35 EUR for the 1 TB drive which is kind of insane when similar drives at local stores cost 110 EUR right now - I remember when those Seagate Barracuda drives were about 40-50 EUR brand new!
For the US, there's also goHardDrive, though I haven't used them myself: https://www.goharddrive.com/category-s/69.htm
I still use SSDs for boot drives, video games and also fragmented data (like Garage for something S3 compatible, which chunks the files), but for backups and longer term storage HDDs are great.
agumonkey|15 days ago