top | item 45602009

(no title)

wuschel | 4 months ago

I had to check for data integrity due to a recent system switch, and was surprised not to find any bitrot after 4y+.

It took ages to compute and verify those hashes between different disks. Certainly an inconvenience.

I am not sure a NAS is really the right solution for smaller data sets. An SSD for quick hashing and a set of N hashed cold storage HDDs - N depends on your appetite for risk - will do.

discuss

order

mrmlz|4 months ago

I've hosted my own data for twenty something years - and bitrot occurs but it is basically caused by two things.

1) Randomness <- this is rare 2) HW-failures <- much more common

So if you catch hw-failures early you can live a long life with very little bitrot... Little =! none so zfs is really great.

wuschel|4 months ago

Don’t get me wrong: IMHO a ZFS mirror setup sounds very tempting, but its strength lie in active data storage. Due to the rarity of bitrot I would argue it can be replaced with manual file hashing (and replacing, if needed) and used in cold storage mode for months.

What worries me more than bitrot is that consumer disks (with enclosure, SWR) do not give access to SMART values over USB via smartctl. Disk failures are real and have strong impact on available data redundancy.

Data storage activities are an exercise in paranoia management: What is truly critical data, what can be replaced, what are the failure points in my strategy?