What a great and valuable post, especially since this info is the result of talking to the APFS team at WWDC, and has not been published anywhere else yet.
Of particular interest (to me) was the "Checksums" section:
Notably absent from the APFS intro talk was any mention of
checksums....APFS checksums its own metadata but not user data.
...The APFS engineers I talked to cited strong ECC protection
within Apple storage devices. Both flash SSDs and magnetic media
HDDs use redundant data to detect and correct errors. The
engineers contend that Apple devices basically don’t return
bogus data.
That is utterly disappointing. SSDs have internal checksums, sure, but there are so many different ways and different points at which a bit can be flipped.
It's hard for me to imagine a worse starting point to conceive a new filesystem than "let's assume our data storage devices are perfect, and never have any faulty components or firmware bugs".
ZFS has a lot of features, but data integrity is the feature.
I get that maybe a checksumming filesystem could conceivably be too computationally expensive for the little jewelry-computers Apple is into these days, but it's a terrible omission on something that is supposed to be the new filesystem for macOS.
I agree; no checksumming of user data is very disappointing. If there were performance issues, they could build checksumming into the filesystem, but make it a volume-specific option. No checksumming on the watch, strong integrity guarantees on the Mac Pro.
Their filesystem goals are in some ways consistent with Apple's (marketing) vision: Users would never have terabyte libraries of anything, as the various iServices would (should) be hosting that stuff in the cloud (where one presumes it is stored on a filesystem that actually includes data integrity). Since users won't be storing much of anything locally, Apple needn't care too much about data integrity. This is of course, nonsense.
The idea that Apple's storage devices are error-free is arrogant--but even assuming that were true, there can still be bit errors in the SATA/PCI bus, errors in memory, race conditions, gamma rays, etc. Apple uses ECC memory on their Mac Pro, so obviously someone still believes that sort of thing is possible.
Talking to the Apple engineers it really didn't seem to be an issue of computation. They seemed genuine in their belief that they could solve data integrity with device qualification. While I asked them 100 questions they asked me 2: had I ever actually seen bit rot (yes), and what kind of drives did we ship with the ZFS Storage Appliance (mostly 7200 nearline drives).
A lot of the CoW patents, WAFL, snapshot patents that Network Appliance filed in the late 1990s have expired, or are expiring this year.
For example, https://www.google.com/patents/US6289356 was filed in 1998, so I presume it's expiring fairly soon. Given that some of the original lawsuits were Network Appliance suing Sun/Oracle, I'm wondering how much of a role this played in the timing of the release of these features? After all, Apple could pretty much pick a window to release a new file system - nothing special about 2016, that they couldn't have done this in 2015 or 2017...
Which makes me wonder if there are data integrity patents that will expire, and at such time, Apple can now drop the functionality into APFS. After all, they did say during their presentation, that the flexibility of the data format is one of the key design features of APFS.
> The engineers contend that Apple devices basically don’t return bogus data.
It's much easier to pretend that this is the case when the file system isn't verifying it.
Checksumming would probably expose problems that would otherwise go unnoticed by users or be blamed on computer gremlins. It's hard to say if doing the "correct" thing here would improve the subjective user experience. Maybe putting on airs of infallibility is the more profitable route.
Agreed. What an arrogant attitude especially considering their Macbook Pro (2015) recently had a corruption bug which necessitated a firmware update:
https://support.apple.com/kb/DL1830?locale=en_GB
Good checksumming to detect bit rot is exactly what is needed since as an owner of said laptop I have NO idea whether any of my data was affected.
If Apple want to say 'the majority of our devices are mobile and checksumming puts a large performance overhead' then that's one thing. But to claim it's not needed is just plain wrong and makes me worry that Apple's product managers sit in an echo chamber hearing only what they want to hear.
> The engineers contend that Apple devices basically don’t return bogus data.
Holy shit.
I guess that explains why my Mac recently had a bunch of daemons burning all cpu crashing repeatedly in a tight loop when getting sqlite errors on a db in ~/Library. Cause disk corruption never happens.
Hmm, I guess like there is the "sufficiently smart compiler" falacy, there is now the "sufficently reliable hardware" falacy.
But on another level, I guess if hardware fails, then well, you buy more hardware, which is good for Apple. Presumably people who bought in the past from Apple won't turn around and buy an Acer or HP laptop. They'll still buy Apple.
When all you have is Apple’s Disk Utility.app, all storage media is perfect. That was irony. Truth is hard drives can have more than 30 bad blocks and still have a verified S.M.A.R.T. status in their app.
I recommend sending every file system engineer on a year-long journey as a traveling system integrator.
i think it makes a lot of sense. turning on these kinds of checks can be scary. the current situation is mostly no-one is effected by bit-rot. this is probably because when it rarely happens it flips some bits that don't really matter anyway. but as soon as you turn on checksumming in software without any automatic error correction people are going to start freaking out when their files become inaccessible or they have to jump through some hoops to access the 'corrupted' file which looks entirely fine to them anyway.
same deal with some heap protections. say you are running a kernel which doesn't have byte patterns to detect heap overflows or reuse after free. maybe you have some heap overflows which because of their nature never cause any corruption but now you turn on heap protections and peoples kernels are getting more panics :/
Perhaps you (or Apple) would still be able to achieve the checksum feature by a smart choice of encryption algorithm?
APFS has file level encryption, so you would in theory be able to detect a flip by selecting an encryption algorithm that gives error upon decrypting modified data. I could see this being worked into apps_fsck at some point.
A similar case could be made for adding it into the compression algorithm, which the OP thinks will be coming to APFS later, popular algorithms such as deflate already have this built in.
Checksums are usually very fast to compute as the ARM CPUs of any modern phone have crypto engines, and their laptops do as well. I think trading data protection for performance reasons would be pretty irrational.
> ZFS has a lot of features, but data integrity is the feature.
And in the Sun era you might be prepared to bet your business on not being on the wrong end of a lawsuit from the owner of the various patents and copyrights around Sun IP.
Only a completely insane person would argue that's a good idea now.
> I get that maybe a checksumming filesystem could conceivably be too computationally expensive for the little jewelry-computers Apple is into these days, but it's a terrible omission on something that is supposed to be the new filesystem for macOS.
Checksumming has another cost that isn't immediately obvious. Suppose you write to a file and the writes are cached. Then the filesystem starts to flush to disk. On a conventional filesystem, you can keep writing to the dirty page while the disk DMAs data out of it. On a checksumming filesystem, you can't: you have to compute the checksum and then write out data consistent with the checksum. This means you either have to delay user code that tries to write, or you have to copy the page, or you need hardware support for checksumming while writing.
On Linux, this type of delay is called "stable pages", and it destroys performance on some workloads on btrfs.
For desktop computing, I'll take data integrity over good 'performance' any day. The use-cases for iDevices might be different, coloring Apples perspective.
Slightly worried by the vibe that comes off this. "I asked him about looking for inspiration in other modern file systems ... he was aware of them, but didn’t delve too deeply for fear, he said, of tainting himself". And (to paraphrase): 'bit-rot? What's that?'.
I would have hoped that a new filesystem with such wide future adoption would have come from a roomful of smart people with lots of experience of (for example) contributing to various modern filesystems, understanding their strengths and weaknesses, and dealing with data corruption issues in the field. This doesn't come across that way at all.
Given Dominic's other output, I'm going to believe there's more to the story because he didn't strike me as someone who would actively ignore past innovations. I know it's a popular concept to NIH stuff when devs believe they know enough to build it, but so much stuff is just built poorly without consideration for existing designs and it shows in the poor software we have to live with.
> With APFS, if you copy a file within the same file system (or possibly the same container; more on this later), no data is actually duplicated. [...] I haven’t see this offered in other file systems [...]
To my knowledge, this is what cp --reflink does on GNU/Linux on a supporting filesystem, most notably btrfs, and has been doing by default in newer combinations of the kernel and GNU coreutils.
This guy seems too well-informed and experienced in the domain to miss something so obvious, though. So what am I missing?
Also interesting to me is the paragraph about prioritizing certain I/O requests to optimize interactive latency: On Linux this is done by the I/O scheduler, exchangable and agnostic to the filesystem. Perhaps greater insight into the filesystem could aid I/O scheduling (this has been the argument for moving RAID code into filesystems as well, though, which APFS opts against) -- hearing a well-informed opinion on this point would be interesting. Unless this post gets it wrong and I/O scheduling isn't technically implemented in APFS either.
It seems like this perspective might be one written from within a Solaris/ZFS bubble and further hamstrung by macOS' closed-source development model. Which is interesting in light of the Giampaolo quote about intentionally not looking closely at the competition, either.
This guy (i.e. me) wasn't aware of this functionality in btrfs. Are reflinks commonly used? Yes, I know more about ZFS than the other filesystems mentioned.
Note mv tries reflink by default (when moving files across BTRFS subvols) since doesn't need a separate copy of the data. Now as storage systems evolve there are less guarantees that one does get multiple copies, with deduplicating at lower layers etc., and therefore cp may change at some stage to reflinking by default, especially as clone_file_range() moves to the VFS level. Actual data redundancy would then achieved at a higher level with separate file systems, devices, data centers, ... where arguably it needs to happen now anyway.
In my opinion, APFS does not seem to improve upon ZFS in several key areas (compression, sending/receiving snapshots, dedup, etc.). Apple is reimplementing many features already implemented in OpenZFS, btrfs (which itself reimplemented a lot of ZFS features), BSD HAMMER, etc.
Maybe extending one of these existing filesystems to add any functionality Apple needs on top of its existing features (and, hopefully, contributing that back to the open source implementation) would cost more person-hours than implementing APFS from scratch. Maybe not.
Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices. Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+. It just seems wasteful of everyone's time to me.
>and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices.
What operating systems describe tremselves as being "compatible with Apple devices"
> Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+.
Who else actually "supports" HFS+ ? Sure there are linux "ports" based on the spec but nobody claims them as being "supported".
Apple would have had to continue supporting HFS+ whether they chose to implement ZFS, btrfs or HAMMER.
>It just seems wasteful of everyone's time to me.
I don't know how Apple writing their own filesystem is wasteful of anybody else's time ( except possibly Apple's and/or Disk utility software for vendors for OS X)
Classic comic, but I don't think it applies. APFS looks intended to solve Apple's product problems really well, and it doesn't even try to be a filesystem for everyone.
Apple has said from time to time that they're all about owning and controlling the key technologies that go into their products. APFS makes a lot of sense from that perspective, and this seems one of those cases where going their own way is better than importing someone else's constraints. ZFS on an Apple Watch? LOL.
APFS is targeted to run on low-powered devices like the Apple Watch. It may be that the alternatives cannot be made suitable for such devices: ZFS is famously memory hungry, HAMMER says it's not designed for < 50GB devices.
Is ZFS lightweight enough to run on the watch? Their ultimate goal was to have a file system efficient enough and flexible enough to run across all of their OS's - MacOS, iOS, tvOs, and watchOS
> Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices.
On the other hand, if Apple decides to open source the APFS implementation (hard to tell what their plans are from current statements, but I'm holding out hope), it'll probably be under a permissive license that allows porting to Linux. The implementation is in C (not C++) so porting is probably generally feasible. Compare to ZFS, which, even if some distros have finally started shipping it, will never quite be free of licensing issues unless Oracle does a 180.
For example, my 1TB SSD includes 1TB (2^30 = 1024^3) bytes of flash but only reports 931GB of available space, sneakily matching the storage industry’s self-serving definition of 1TB (1000^3 = 1 trillion bytes).
Great article, but a couple nitpicking corrections (which seem appropriate for a storage article)
Per: https://en.wikipedia.org/wiki/Terabyte - Terabyte is 1000^4, not 1000^3.
Also It's been 6+ years since we all agreed that TiB means 2^40 or 1024^4, and TB means 10^12. Indeed, only in the case of memory does "T" ever mean 2^40 anyways. It's always been the case that in both data rates, as well as storage, that T means 10^12. This convention is strong enough that we most of us just have thrown up our hands and agree when referring to DRAM memory, that Terabyte will mean 1024^4, and 1000^4 everywhere else.
Indeed, in the rare case where someone uses TiB to refer to a data rate, they are almost without exception incorrectly using it, and, they actually mean TB.
> Also, APFS removes the most common way of a user achieving local data redundancy: copying files. A copied file in APFS actually creates a lightweight clone with no duplicated data.
No, it doesn't. APFS supports copying files, if you want that. It's just that the default in Finder is to make a “clone” (copy-on-write).
Fair enough; and right now cp doesn't use the fast clone functionality, but it assuredly will. I'm not sure 'cat <file >file.dup' is reasonable for most users.
I'm still looking for a widely supported (at least FreeBSD and Linux kernels) filesystem for external drives to carry around that doesn't have the FAT32 limitations. There's exFAT but no stable and supported implementation. Then there's NTFS, but that's also not 100% reliable in my experience when used through FUSE (NTFS-3G). I've considered UFS but that also was a no go. I'm hopeful for lklfuse[1] that also runs on FreeBSD and givess access to ext4, xfs, etc. in a way like Rump and allows you to use the same drivers on FreeBSD. I'm cautious though, given that I don't want corrupted data I might notice too late. Let's see if lklfuse provides LUKS as well, otherwise Dragonfly's LUKS implementation might need to be ported to FreeBSD or something like that. External drives one might lose need to be encrypted.
Thanks for this! I didn't realize (or had forgotten) LUKS had been ported to Dragonfly. Also you touch upon my #1 frustration with APFS without really knowing anything about it: simple portability.
The file-level deduplication [1] is interesting. Not being a filesystem expert, this sounds like it fulfills a similar usecase to snapshots [2]. Or am I reading this wrong?
NTFS Shadow Copies are more like LVM/ZFS snapshots than APFS's file-level CoW snapshots, in that they both operate on the entire volume at block-level, rather than having a per-file level of granularity.
There are other FSes that allow the behavior that APFS is demonstrating - look at OCFS2 and Btrfs, both of which allow you to do cp --reflink.
I think the value of this new proprietary filesystem is limited, since you can't run it on servers (Apple does not make servers anymore). Also, compatibility/porting issues may become a problem if you build your software for it.
[+] [-] veidr|9 years ago|reply
Of particular interest (to me) was the "Checksums" section:
That is utterly disappointing. SSDs have internal checksums, sure, but there are so many different ways and different points at which a bit can be flipped.It's hard for me to imagine a worse starting point to conceive a new filesystem than "let's assume our data storage devices are perfect, and never have any faulty components or firmware bugs".
ZFS has a lot of features, but data integrity is the feature.
I get that maybe a checksumming filesystem could conceivably be too computationally expensive for the little jewelry-computers Apple is into these days, but it's a terrible omission on something that is supposed to be the new filesystem for macOS.
[+] [-] isomorphic|9 years ago|reply
Their filesystem goals are in some ways consistent with Apple's (marketing) vision: Users would never have terabyte libraries of anything, as the various iServices would (should) be hosting that stuff in the cloud (where one presumes it is stored on a filesystem that actually includes data integrity). Since users won't be storing much of anything locally, Apple needn't care too much about data integrity. This is of course, nonsense.
The idea that Apple's storage devices are error-free is arrogant--but even assuming that were true, there can still be bit errors in the SATA/PCI bus, errors in memory, race conditions, gamma rays, etc. Apple uses ECC memory on their Mac Pro, so obviously someone still believes that sort of thing is possible.
[+] [-] ahl|9 years ago|reply
[+] [-] ghshephard|9 years ago|reply
For example, https://www.google.com/patents/US6289356 was filed in 1998, so I presume it's expiring fairly soon. Given that some of the original lawsuits were Network Appliance suing Sun/Oracle, I'm wondering how much of a role this played in the timing of the release of these features? After all, Apple could pretty much pick a window to release a new file system - nothing special about 2016, that they couldn't have done this in 2015 or 2017...
Which makes me wonder if there are data integrity patents that will expire, and at such time, Apple can now drop the functionality into APFS. After all, they did say during their presentation, that the flexibility of the data format is one of the key design features of APFS.
[+] [-] tveita|9 years ago|reply
It's much easier to pretend that this is the case when the file system isn't verifying it.
Checksumming would probably expose problems that would otherwise go unnoticed by users or be blamed on computer gremlins. It's hard to say if doing the "correct" thing here would improve the subjective user experience. Maybe putting on airs of infallibility is the more profitable route.
[+] [-] leonroy|9 years ago|reply
Good checksumming to detect bit rot is exactly what is needed since as an owner of said laptop I have NO idea whether any of my data was affected.
If Apple want to say 'the majority of our devices are mobile and checksumming puts a large performance overhead' then that's one thing. But to claim it's not needed is just plain wrong and makes me worry that Apple's product managers sit in an echo chamber hearing only what they want to hear.
[+] [-] asveikau|9 years ago|reply
Holy shit.
I guess that explains why my Mac recently had a bunch of daemons burning all cpu crashing repeatedly in a tight loop when getting sqlite errors on a db in ~/Library. Cause disk corruption never happens.
[+] [-] rdtsc|9 years ago|reply
But on another level, I guess if hardware fails, then well, you buy more hardware, which is good for Apple. Presumably people who bought in the past from Apple won't turn around and buy an Acer or HP laptop. They'll still buy Apple.
[+] [-] nier|9 years ago|reply
I recommend sending every file system engineer on a year-long journey as a traveling system integrator.
[+] [-] ot|9 years ago|reply
[+] [-] benmmurphy|9 years ago|reply
same deal with some heap protections. say you are running a kernel which doesn't have byte patterns to detect heap overflows or reuse after free. maybe you have some heap overflows which because of their nature never cause any corruption but now you turn on heap protections and peoples kernels are getting more panics :/
[+] [-] speleding|9 years ago|reply
APFS has file level encryption, so you would in theory be able to detect a flip by selecting an encryption algorithm that gives error upon decrypting modified data. I could see this being worked into apps_fsck at some point.
A similar case could be made for adding it into the compression algorithm, which the OP thinks will be coming to APFS later, popular algorithms such as deflate already have this built in.
[+] [-] codemac|9 years ago|reply
[+] [-] rodgerd|9 years ago|reply
And in the Sun era you might be prepared to bet your business on not being on the wrong end of a lawsuit from the owner of the various patents and copyrights around Sun IP.
Only a completely insane person would argue that's a good idea now.
[+] [-] mahyarm|9 years ago|reply
[+] [-] KaiserPro|9 years ago|reply
[+] [-] amelius|9 years ago|reply
[+] [-] amluto|9 years ago|reply
Checksumming has another cost that isn't immediately obvious. Suppose you write to a file and the writes are cached. Then the filesystem starts to flush to disk. On a conventional filesystem, you can keep writing to the dirty page while the disk DMAs data out of it. On a checksumming filesystem, you can't: you have to compute the checksum and then write out data consistent with the checksum. This means you either have to delay user code that tries to write, or you have to copy the page, or you need hardware support for checksumming while writing.
On Linux, this type of delay is called "stable pages", and it destroys performance on some workloads on btrfs.
[+] [-] sangnoir|9 years ago|reply
[+] [-] gmac|9 years ago|reply
I would have hoped that a new filesystem with such wide future adoption would have come from a roomful of smart people with lots of experience of (for example) contributing to various modern filesystems, understanding their strengths and weaknesses, and dealing with data corruption issues in the field. This doesn't come across that way at all.
[+] [-] cm3|9 years ago|reply
[+] [-] kev009|9 years ago|reply
[+] [-] sho_hn|9 years ago|reply
> With APFS, if you copy a file within the same file system (or possibly the same container; more on this later), no data is actually duplicated. [...] I haven’t see this offered in other file systems [...]
To my knowledge, this is what cp --reflink does on GNU/Linux on a supporting filesystem, most notably btrfs, and has been doing by default in newer combinations of the kernel and GNU coreutils.
This guy seems too well-informed and experienced in the domain to miss something so obvious, though. So what am I missing?
Also interesting to me is the paragraph about prioritizing certain I/O requests to optimize interactive latency: On Linux this is done by the I/O scheduler, exchangable and agnostic to the filesystem. Perhaps greater insight into the filesystem could aid I/O scheduling (this has been the argument for moving RAID code into filesystems as well, though, which APFS opts against) -- hearing a well-informed opinion on this point would be interesting. Unless this post gets it wrong and I/O scheduling isn't technically implemented in APFS either.
It seems like this perspective might be one written from within a Solaris/ZFS bubble and further hamstrung by macOS' closed-source development model. Which is interesting in light of the Giampaolo quote about intentionally not looking closely at the competition, either.
[+] [-] ahl|9 years ago|reply
[+] [-] pixelbeat|9 years ago|reply
[+] [-] e12e|9 years ago|reply
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] idorosen|9 years ago|reply
Maybe extending one of these existing filesystems to add any functionality Apple needs on top of its existing features (and, hopefully, contributing that back to the open source implementation) would cost more person-hours than implementing APFS from scratch. Maybe not.
Either way, we will now have yet another filesystem to contend with, implement in non-Darwin kernels (maybe), and this adds to the overall support overhead of all operating systems that want to be compatible with Apple devices. Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+. It just seems wasteful of everyone's time to me.
Also: https://xkcd.com/927/
[+] [-] vvhn|9 years ago|reply
What operating systems describe tremselves as being "compatible with Apple devices"
> Since the older versions of macOS (OSX) don't support APFS, only HFS+, this means Apple and others will also have to continue supporting HFS+.
Who else actually "supports" HFS+ ? Sure there are linux "ports" based on the spec but nobody claims them as being "supported". Apple would have had to continue supporting HFS+ whether they chose to implement ZFS, btrfs or HAMMER.
>It just seems wasteful of everyone's time to me.
I don't know how Apple writing their own filesystem is wasteful of anybody else's time ( except possibly Apple's and/or Disk utility software for vendors for OS X)
>Also: https://xkcd.com/927/
The standard is the interface ( POSIX / SUS ) and unless APFS breaks that how is this applicable ?
[+] [-] mindajar|9 years ago|reply
Apple has said from time to time that they're all about owning and controlling the key technologies that go into their products. APFS makes a lot of sense from that perspective, and this seems one of those cases where going their own way is better than importing someone else's constraints. ZFS on an Apple Watch? LOL.
[+] [-] millstone|9 years ago|reply
[+] [-] scarface74|9 years ago|reply
[+] [-] comex|9 years ago|reply
On the other hand, if Apple decides to open source the APFS implementation (hard to tell what their plans are from current statements, but I'm holding out hope), it'll probably be under a permissive license that allows porting to Linux. The implementation is in C (not C++) so porting is probably generally feasible. Compare to ZFS, which, even if some distros have finally started shipping it, will never quite be free of licensing issues unless Oracle does a 180.
[+] [-] matthewmcg|9 years ago|reply
See http://arstechnica.com/apple/2009/10/apple-abandons-zfs-on-m...
[+] [-] lanna|9 years ago|reply
[+] [-] ghshephard|9 years ago|reply
Great article, but a couple nitpicking corrections (which seem appropriate for a storage article) Per: https://en.wikipedia.org/wiki/Terabyte - Terabyte is 1000^4, not 1000^3.
Also It's been 6+ years since we all agreed that TiB means 2^40 or 1024^4, and TB means 10^12. Indeed, only in the case of memory does "T" ever mean 2^40 anyways. It's always been the case that in both data rates, as well as storage, that T means 10^12. This convention is strong enough that we most of us just have thrown up our hands and agree when referring to DRAM memory, that Terabyte will mean 1024^4, and 1000^4 everywhere else.
Indeed, in the rare case where someone uses TiB to refer to a data rate, they are almost without exception incorrectly using it, and, they actually mean TB.
[+] [-] TazeTSchnitzel|9 years ago|reply
No, it doesn't. APFS supports copying files, if you want that. It's just that the default in Finder is to make a “clone” (copy-on-write).
[+] [-] ahl|9 years ago|reply
[+] [-] cm3|9 years ago|reply
[1] https://www.freshports.org/sysutils/fusefs-lkl/
[+] [-] drvdevd|9 years ago|reply
[+] [-] niftich|9 years ago|reply
Is NTFS's shadow copy like Snapshots?
[1] http://dtrace.org/blogs/ahl/2016/06/19/apfs-part3/#apfs-clon...
[2] http://dtrace.org/blogs/ahl/2016/06/19/apfs-part2/#apfs-snap...
[+] [-] rincebrain|9 years ago|reply
There are other FSes that allow the behavior that APFS is demonstrating - look at OCFS2 and Btrfs, both of which allow you to do cp --reflink.
[+] [-] amelius|9 years ago|reply
[+] [-] maureenash|9 years ago|reply
[deleted]