> “In a typical small business/home NAS environment, workloads tend to be bursty in nature, leaving sufficient idle time for garbage collection and other maintenance operations.”
Only up until the array needs to be rebuilt (eg replacing a drive). At which point the workload is the literal opposite of "bursty with idle time". :(
My understanding is that it removes the ability to do random writes, since you'll disturb the nearby tracks. Like drawing pen thickness rings with a sharpie by overlapping them.
Instead you read data in, modify it, then write it out, like an extreme version of updating a byte in a cache line.
Your ability to read is more precise, so you have no trouble reading data laid out in this way.
Large contiguous writes are also pretty quick. I've owned one, and if you stick to large files it's fine. Pathological workloads like creating millions of small files can take days.
Rebuilding is not random i/o though, it goes sequentially and that's what SMRs are good at.
Still, I did recently reshape from 4x6TB raid5 to 5x6TB raid6. The new drive was SMR WD Red, it took 12 days and I was quite pissed when I found out the SMR bit. It wasn't in the datasheet, if I knew upfront, I would never bought it.
That's a problem. I have been using and recommending WD to everyone who asks me what drive they should buy (based on backblaze data and 20 years of personal experience) for their private and everyday usage. Even though these are NAS drives it tarnishes the brand.
Trust is a bitch - so hard to gain it, and very very easy to lose by doing shit like this. I was buying WD/HGST drives for work. Now, I have to test everything I buy before deploying them dammit.
It seems pretty calculated to reduce costs while deceiving the buyer by omitting or having an incorrect spec sheet.
Would a class action suit help to prevent this kind of action in the future? Perhaps the punitive fine will make the cost calculations that led to this decision different since they will need to include the risk of judgement in the costs.
I would imagine not. The class action would settle, most consumers would get a trivial sum in the form of a check, and the litigating firm would get a large payout. The settlement would result in no one taking responsibility for anything.
Aside from the other mentioned issues, a significant part is also the lack of retesting. WD Red drives aren't a fancy new thing and have been on the market for years now. As a result a test of the same drives is unlikely to give you a whole lot of revenue, since it's a known old product. People don't usually care a lot about yet another test with the same results. As a result retesting isn't really worth your time _unless_ you find something significantly changed (like in this case). Also running a useful test is non-trival as you'd need to source a significant amount of platforms, comparision products and setup a testing environment for it. Testing a brand new (CPU|GPU|SSD) instead would be much more likely to attract new readers and increase revenue.
Manufacturers have slowly started realizing this circumstance and are obviously able to exploit it by making products cheaper to manufacture down the line.
Yes, you will notice this, even if you do a simple write test with only big writes at full speed. After a while (about 2h in our tests) these drives become "tired". You need to leave them be for a few hours and then they are fine again.
An added problem is that the usual candidate file systems (ext4, xfs,...) do not combine well with SMR.
Some people might be able to put these drives to good use, but only if they know up front that it's this kind of drive they are getting.
Yes. I remember two years back when a small (really) independent french journal called CPC Hardware was able to publish a full review of the latest AMD CPU a month before everyone else in a bimonthly paper issue: they simply sourced the chip on eBay instead of signing NDAs like literally every other publication.
I dont know of any independent review site besides BackBlaze.
Most review sites receive product samples from the manufacturer. The linked article shows that the disk manufacturers cannot be trusted.
I bet they did not send out DM-SMR drives to review sites.
"The Synology forum poster said he called WD support to ask if the drive was an SMR or conventionally recorded drive..... the higher team contacted me back and informed me that the information I requested about whether or not the WD60EFAX was a SMR or PMR would not be provided to me. They said that information is not disclosed to consumers"
I don't know or care about shingled, thatch, whatever, but I am not going to entrust my irreplaceable data to a vendor that refuses to tell me or anyone else what they are selling.
My understanding is that the SMR drives are like SSD to a degree that they needs rather intelligent firmware to place the information on the disk.
At the moment everything is sitting behind the bogo SATA disk interface which was never designed for this type of storage - rapid burst writes to the 20GB onboard buffer area and than very slow I/O as it is emptied out.
Surely a native OS interface would be better for these drives.
It's much worse than that. For SSD, it's just the blocks need to be zeroed before being writing to, which is not even an issue if there is lot of free space.
But for SMR, the reading track is narrower than writing track. For example, reading track's width is 1 unit, writing track's width is 2 unit, there is a 1 unit overlap between two adjacent tracks.
And here is how it works, assuming writing sequentially:
1. Start with blank disk, and write to track #1. This is happy case, as no data there, we can simply write to it.
2. Write to track #2. Still can simply write to it. However, this will overwrite lower half of track #1, but as long as reading only need 1 unit, track #1 record is still good.
3. Write to the following tracks one by one, all good, same story with step #2, over writing half of previous track.
4. Repeat until track #n.
5. Okay, all of a sudden, now we need to change something in track #1. What would happen now: we cannot simply overwrite the stuff in track #1 as this will overwrite the top half of track #2, which is the reading track of track #2. So we'll have to read out track #2 and and write track #1 and then write back track #2, which in turn has to read out track #3 and write track #2 and write back track #3...
6. Repeat the previous operation until all impacted tracks are written back.
In reality, the situation could be slightly better: if the disk is almost empty, the firmware can always find somewhere to write to without reading out and writing back. But if the disk is relatively full, then the case could be much worse: every writing could technically trigger another reading out and writing back operation. So how long would it take to finish a writing operation can hardly be predicted.
The worse case is that the disk is full and the writing is targeting whole track #1, then almost the entire disk need to be refreshed so that it can complete the operation...
It's worse than that. SMR drives can't do random writes without rewriting large sections. So they are entirely unsuitable for general purpose workloads.
The Toshiba and Seagate NAS lines seem well organized and better alternatives these days. And I guess this is all the more reason to get one drive per manufacturer for the typical ~4 drives in a home NAS. Between Seagate/Toshiba/WD/HGST there should be enough variation to still protect against this stuff.
Edit: Seagate is doing the exact same thing [0], although not confirmed for NAS drives... yet. HGST is owned by WD (although the 3.5" side apparently went to Toshiba). So that leaves Toshiba.
It sucks to have to qualify drives (again?) in small businesses, thought this was a solved problem. It's more offensive they're doing it with the smaller capacities, too.
(I should note SSDs don't stand up to manufacturer's claims in error scenarios, too. But that's fairly well known I think?)
Seagate drives are notoriously poor quality, in addition to well published, large sample and long running studies by Backblaze - I ran a number of decent sized storage clusters for many years and by far found Seagate drives the most unreliable (including Seagate a rebranded as HPE etc...)
Maybe they refreshed their 2-6TB line recently and not the 8-14TB and it'll come later?
Maybe the market for 8-14TB is different enough that it warrants CMS? For instance maybe the 2-6TB is for hobbyists/small companies who might not realize the drives behave differently while the larger and more expensive drives are more often use in large companies with dedicated IT who would realize something is off?
Given that TFA mainly mentions 2-6TB several times I doubt they got it wrong.
My guess is the want to further segment the markets so they can improve price discrimination, but they don’t want to sell a “lite” or “value” drive in the NAS segment, so they change the base model(s) to be shittier and accept the bad PR.
That way they can come out with a more expensive PMR SKU in the NAS series. SSD manufacturers did a lot of the same where features were stripped from consumer drives (PLP, DRAT/DZAT, etc.), segmented into a new market (notice the NAS SSDs?), and sold back to us as more expensive SKU.
I was wondering about this too, and I think it’s for cost-cutting on the lowest-margin products in the range.
SMR is going to reduce platter size and quality requirements for a given size in GB (because the data is packed more densely). A given capacity can require fewer platters/sides, or a blemished platter can now be used where it couldn’t previously.
For all the "can't trust HGST because its owned by WD" comments: I don't think this is the case, as HGST is still selling drives that are explicitly declared to be PMR is the datasheets, e.g. here's the HGST Ultrastar DC520 [0] which includes PMR as one of its selling points.
Eh I still wouldn't want to give money to a company that's segmented itself into a "lie to the customer" branch and a "don't lie to the customer" branch. The former will poison the latter eventually.
I would think that one would typically expect NAS drives in particular not to use SMR, especially since SMR is known to be problematic in RAID setups, etc. Not to mention the write performance issues.
It's so frustrating that all that was needed to avoid most of this was some transparency. At least list in the specs that they have SMR. It's essentially lying by omission.
But I'd probably go a step further and say that marketing an SMR drive as a NAS drive is user-hostile and destined to cause problems
Agreed. I will add that the general trend seems to assume that user is a functional idiot and keep actual specs either hidden away or difficult to get to. It is hard to make informed decisions in such environment.
The fact that WD is refusing to tell customers basic facts about their products even when called is a huge red flag. There is no legitimate reason to refuse to tell customers about whether or not you’re using SMR on a given drive.
A funny thing about device managed HMR (DMSMR) is that they extensively rely on error correction codes to recover data from overwritten tracks, and loose near all of SMR density gain to extra coding.
Why DMSMR became a thing in the first place? Windows! And non-upgradeable proprietary storage array hardware.
It seems that business people in those storage companies have put big money into SMR, but only realised that an extremely big chunk of the market cannot adopt SMR after drives shipped. And so they simply slapped a cheap and easy firmware hack on top of SMR drives to make them work with non-SMR aware software.
What exactly do companies gain by such deceptive obfuscation and secrecy of the details about the products they sell? Especially when some testing can reveal the truth easily. It seems to be a bit of an industry trend in general, since very old HDD datasheets would specify even things like how many tracks per inch the platters have, how many heads, sectors per track, spindle motor spin-up time, etc. Somewhat recently, the WD Greens famously did not specify their rotational speed.
As I alluded to above, HDDs aren't the only things affected by this trend; SSDs, or more precisely NAND flash manufacturers, have become equally secretive about endurance and data retention. Older datasheets were freely available and specified the number of cycles each block could be guaranteed to be rewritten and how long the data would stay, but datasheets for newer flash are basically all NDA-only and even then you won't get the exact details. At least in that case, I suspect they are trying to hide the inconvenient truth of decreasing reliability; software workarounds can only go so far.
"There are three ways to make a living in this business: be first, be smarter, or cheat." -Margin Call
It feels like the first two are increasingly taken, so businesses are lazily switching to various versions of #3. I don't know where the regulators and punishment for doing that have gone.
I don't understand enough of the technical details in the article - can someone please explain what this means in practice?
I use WD Reds in my Synology NAS in a SHR configuration for daily backups. Should I be worried? Should I replace the disks with other models? What are the risks if I keep the current configuration?
Why are they using SMR on small capacity drives? I thought the driving force behind SMR was for higher capacity? I know cheaper 8TB+ drives use SMR.
Is it so they can use fewer platters to get the same capacity, thus reducing the manufacturing cost of the drive? Or is there some other technical reason to do it on low capacity drives?
Does this mean anything for the Purple line that are specifically for surveillance applications?
My assumption would be that the Purple drives become the default for NAS usage as the surveillance use-case much more closely matches the requirements of a RAID rebuild (a fairly constant write-stream).
My anecdata sample size of 3 has WD Red drives priced very closely to equivalently sized WD Purple drives, with WD Black drives a significant percentage more expensive (40% +).
Does this mean WD Black drives are the last hold-out against SMR for WD brand?
[+] [-] justinclift|5 years ago|reply
Only up until the array needs to be rebuilt (eg replacing a drive). At which point the workload is the literal opposite of "bursty with idle time". :(
[+] [-] reitzensteinm|5 years ago|reply
My understanding is that it removes the ability to do random writes, since you'll disturb the nearby tracks. Like drawing pen thickness rings with a sharpie by overlapping them.
Instead you read data in, modify it, then write it out, like an extreme version of updating a byte in a cache line.
Your ability to read is more precise, so you have no trouble reading data laid out in this way.
Large contiguous writes are also pretty quick. I've owned one, and if you stick to large files it's fine. Pathological workloads like creating millions of small files can take days.
[+] [-] vetinari|5 years ago|reply
Still, I did recently reshape from 4x6TB raid5 to 5x6TB raid6. The new drive was SMR WD Red, it took 12 days and I was quite pissed when I found out the SMR bit. It wasn't in the datasheet, if I knew upfront, I would never bought it.
[+] [-] johnchristopher|5 years ago|reply
[+] [-] reacharavindh|5 years ago|reply
[+] [-] stardude900|5 years ago|reply
[+] [-] prepend|5 years ago|reply
Would a class action suit help to prevent this kind of action in the future? Perhaps the punitive fine will make the cost calculations that led to this decision different since they will need to include the risk of judgement in the costs.
[+] [-] sircastor|5 years ago|reply
[+] [-] dbuder|5 years ago|reply
[deleted]
[+] [-] jl6|5 years ago|reply
[+] [-] sbx320|5 years ago|reply
Manufacturers have slowly started realizing this circumstance and are obviously able to exploit it by making products cheaper to manufacture down the line.
[+] [-] toolslive|5 years ago|reply
Some people might be able to put these drives to good use, but only if they know up front that it's this kind of drive they are getting.
[+] [-] Aissen|5 years ago|reply
https://www.guru3d.com/news-story/cpc-hardware-amd-ryzen-7-2...
[+] [-] tibbydudeza|5 years ago|reply
Reviews these days are rather all about GPU's and CPU's.
[+] [-] skinkestek|5 years ago|reply
[+] [-] duxup|5 years ago|reply
Were these new enough that anyone bothered?
[+] [-] frankharv|5 years ago|reply
[+] [-] syshum|5 years ago|reply
[+] [-] basicplus2|5 years ago|reply
[+] [-] gowld|5 years ago|reply
[+] [-] BadBadJellyBean|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] tibbydudeza|5 years ago|reply
At the moment everything is sitting behind the bogo SATA disk interface which was never designed for this type of storage - rapid burst writes to the 20GB onboard buffer area and than very slow I/O as it is emptied out.
Surely a native OS interface would be better for these drives.
[+] [-] mrjin|5 years ago|reply
But for SMR, the reading track is narrower than writing track. For example, reading track's width is 1 unit, writing track's width is 2 unit, there is a 1 unit overlap between two adjacent tracks.
And here is how it works, assuming writing sequentially: 1. Start with blank disk, and write to track #1. This is happy case, as no data there, we can simply write to it. 2. Write to track #2. Still can simply write to it. However, this will overwrite lower half of track #1, but as long as reading only need 1 unit, track #1 record is still good. 3. Write to the following tracks one by one, all good, same story with step #2, over writing half of previous track. 4. Repeat until track #n. 5. Okay, all of a sudden, now we need to change something in track #1. What would happen now: we cannot simply overwrite the stuff in track #1 as this will overwrite the top half of track #2, which is the reading track of track #2. So we'll have to read out track #2 and and write track #1 and then write back track #2, which in turn has to read out track #3 and write track #2 and write back track #3... 6. Repeat the previous operation until all impacted tracks are written back.
In reality, the situation could be slightly better: if the disk is almost empty, the firmware can always find somewhere to write to without reading out and writing back. But if the disk is relatively full, then the case could be much worse: every writing could technically trigger another reading out and writing back operation. So how long would it take to finish a writing operation can hardly be predicted.
The worse case is that the disk is full and the writing is targeting whole track #1, then almost the entire disk need to be refreshed so that it can complete the operation...
[+] [-] nicoburns|5 years ago|reply
[+] [-] isatty|5 years ago|reply
[0] https://github.com/openzfs/zfs/issues/10182
[+] [-] pedrocr|5 years ago|reply
[+] [-] guitarbill|5 years ago|reply
It sucks to have to qualify drives (again?) in small businesses, thought this was a solved problem. It's more offensive they're doing it with the smaller capacities, too.
(I should note SSDs don't stand up to manufacturer's claims in error scenarios, too. But that's fairly well known I think?)
[0] https://blocksandfiles.com/2020/04/15/seagate-2-4-and-8tb-ba...
[+] [-] smcleod|5 years ago|reply
- https://www.backblaze.com/blog/hard-drive-stats-q2-2019/
- https://blocksandfiles.com/2020/02/13/seagate-12tb-drive-fai...
[+] [-] s9w|5 years ago|reply
[+] [-] gsich|5 years ago|reply
[+] [-] zajio1am|5 years ago|reply
This does not make sense - why smaller disks are SMR, while larger ones are CMR? Perhaps they just switched that in comment?
[+] [-] simias|5 years ago|reply
Maybe the market for 8-14TB is different enough that it warrants CMS? For instance maybe the 2-6TB is for hobbyists/small companies who might not realize the drives behave differently while the larger and more expensive drives are more often use in large companies with dedicated IT who would realize something is off?
Given that TFA mainly mentions 2-6TB several times I doubt they got it wrong.
[+] [-] donmcronald|5 years ago|reply
That way they can come out with a more expensive PMR SKU in the NAS series. SSD manufacturers did a lot of the same where features were stripped from consumer drives (PLP, DRAT/DZAT, etc.), segmented into a new market (notice the NAS SSDs?), and sold back to us as more expensive SKU.
[+] [-] throwaheyy|5 years ago|reply
SMR is going to reduce platter size and quality requirements for a given size in GB (because the data is packed more densely). A given capacity can require fewer platters/sides, or a blemished platter can now be used where it couldn’t previously.
[+] [-] duskwuff|5 years ago|reply
No, I think that's correct. It's definitely weird, but it's consistent with what users have observed.
[+] [-] ComputerGuru|5 years ago|reply
[0]: https://documents.westerndigital.com/content/dam/doc-library...
[+] [-] p1necone|5 years ago|reply
[+] [-] kup0|5 years ago|reply
It's so frustrating that all that was needed to avoid most of this was some transparency. At least list in the specs that they have SMR. It's essentially lying by omission.
But I'd probably go a step further and say that marketing an SMR drive as a NAS drive is user-hostile and destined to cause problems
[+] [-] A4ET8a8uTh0|5 years ago|reply
[+] [-] ashtonkem|5 years ago|reply
[+] [-] baybal2|5 years ago|reply
Why DMSMR became a thing in the first place? Windows! And non-upgradeable proprietary storage array hardware.
It seems that business people in those storage companies have put big money into SMR, but only realised that an extremely big chunk of the market cannot adopt SMR after drives shipped. And so they simply slapped a cheap and easy firmware hack on top of SMR drives to make them work with non-SMR aware software.
[+] [-] userbinator|5 years ago|reply
As I alluded to above, HDDs aren't the only things affected by this trend; SSDs, or more precisely NAND flash manufacturers, have become equally secretive about endurance and data retention. Older datasheets were freely available and specified the number of cycles each block could be guaranteed to be rewritten and how long the data would stay, but datasheets for newer flash are basically all NDA-only and even then you won't get the exact details. At least in that case, I suspect they are trying to hide the inconvenient truth of decreasing reliability; software workarounds can only go so far.
[+] [-] JohnJamesRambo|5 years ago|reply
It feels like the first two are increasingly taken, so businesses are lazily switching to various versions of #3. I don't know where the regulators and punishment for doing that have gone.
[+] [-] Paul-ish|5 years ago|reply
[+] [-] kruuuder|5 years ago|reply
I use WD Reds in my Synology NAS in a SHR configuration for daily backups. Should I be worried? Should I replace the disks with other models? What are the risks if I keep the current configuration?
[+] [-] Thev00d00|5 years ago|reply
[+] [-] Osiris|5 years ago|reply
Is it so they can use fewer platters to get the same capacity, thus reducing the manufacturing cost of the drive? Or is there some other technical reason to do it on low capacity drives?
[+] [-] peter_d_sherman|5 years ago|reply
"However, SMR drives are not intended for random write IO use cases because the write performance is much slower than with a non-SMR drive."
Might this be a way to detect SMR usage by allegedly non-SMR drives?
Do a whole lot of random write IO -- and time and statistically sort the results?
[+] [-] BLKNSLVR|5 years ago|reply
My assumption would be that the Purple drives become the default for NAS usage as the surveillance use-case much more closely matches the requirements of a RAID rebuild (a fairly constant write-stream).
My anecdata sample size of 3 has WD Red drives priced very closely to equivalently sized WD Purple drives, with WD Black drives a significant percentage more expensive (40% +).
Does this mean WD Black drives are the last hold-out against SMR for WD brand?