Excellent work - a methodical review like this is exactly what I’ve been looking for in these sorts of open source solutions.
I know it’s not the focus of a code review like this, but I’m interested to hear your views on the general supply chain lifecycle problems inherent to open-source package management platforms. Principally, are vetting processes appropriate to ensure that new formulas refer to the correct source? How does the user gain confidence that their brew update is still referencing a trusted source? What happens when a domain is taken over? How quickly can the team respond to untrusted sources from formulas?
I know these aren’t all Homebrew problems to solve, but they’re important ecosystem considerations.
(These problems also exist in the winget and choco platforms, but less so in commercially supported repos like apt and yum. For me, and many other admins, they are a major concern when it comes to the Windows Store.)
Edit: lastly, in case the homebrew team are watching: an npm-style vulnerability notice would be awesome
I can't emphasize enough how much of a genuine issue this is, especially where package managers are being used on production environments or within CI/CD pipelines. There's enough publicized cases of Chinese CCP operatives gaining pull request access to key packages, and I'm sure many more get discovered that are covered up/not made public. Even just turn over of package ownership from reputable entities to lesser known individuals is of course worrying.
As a SWE/EngMgr turned VC, I'm curious if there's startups or commercial companies providing some kind of assurance here (but also worried the $ TAM for solving this problem probably isn't enough to make it a standalone business).
Before moving to Nix, I was using MacPorts since Homebrew had some...eccentric behavior at the time (didn't work with multi-user setups, owned your /usr/local, lots of "works on my machine" problems from auto-updating and lack of version control, ...). One thing that has always felt insecure about Homebrew to me was the ability to use GitHub (not Git) URLs as ad-hoc packages. I wonder if that is how TOB-BREW-13 worked? That feature of Homebrew has always sounded like a security incident waiting to happen.
In any case, I'd be interested in seeing an audit of Nix on Mac OS. Especially if there is a flaw in how `nix develop` and related commands work.
Funny that you mention it, I also went Homebrew -> MacPorts -> Nix.
Homebrew had analytics and broke versions too often. MacPorts is way more stable, but some niche packages would not build well, and I had terminfo issues with tmux.
Nix allows me to override most of that, and I can share home manager config with my Debian workstation.
Nix doesn't sandbox builds by default on macOS. You can try enabling it yourself with `sandbox = true` in nix.conf, but Things May Break.
The Nix sandbox is also not really meant as a security boundary; there's no effort put into preventing sandbox escapes, and lots of stuff leaks from the host into the sandbox environment. You really want something like gVisor or a full VM if you want to build untrusted packages.
I peeked at the sandbox escape bug and its associated fix: https://github.com/Homebrew/brew/pull/17700/commits/f4e5e0c7.... Is this…correct? Like, this is to prevent it from being interpolated into a sandbox profile and messing things up. How confident are we about this list of characters?
The commit message could have answered that question, but instead, it only repeats what the diff already tells us: "Don't allow special characters in sandbox rule paths". It doesn't explain why, or what's special about the particular characters that it disallows. A better message would have said something like "Prevent certain characters in the path because otherwise ... We need to worry about these specific characters but not others because..."
Even if I wrote this code and knew what it did today, if I come across this it a year from now, I'm left scratching my head: why did I make this change?
I ditched `brew` for `nix` a while back and while the TUI could be more end-user-friendly (to the point that I wrote a wrapper called "ixnay" just so I could do "ixnay install <packagename>" as easily as with brew, https://github.com/pmarreck/ixnay), the overall guarantees make it worth it.
I wish I had known about ixnay earlier! I also got annoyed of the user experience, to the point where I also wrote my own tool, hdn: https://github.com/seasonedfish/hdn
I am a bit surprised this did not highlight major low skill attack surface in Homebrew as compared to almost all Linux and *BSD package managers: Supply chain integrity.
Homebrew maintainers mostly do not sign commits/packages, do not sign reviews/merges, do not verify author/reviewer sigs at compile time, do not reproduce builds in separately controlled CI, do not enforce hardware 2FA on Github.
Every user of brew is only as secure as whichever of hundreds of brew maintainers has the worst opsec today.
Also since dependabot automatically makes commits, you could get a malicious commit into an external project you control, wait for dependabot to make a commit to homebrew to upgrade it, then merge it yourself (as becoming a homebrew maintainer has almost no vetting, just fix a few easy bugs)
You could also just take over one of the expired email domains of a maintainer and send a password reset email to yourself and take over an account of someone on vacation or hiatus.
Can likely get thousands of companies compromised before anyone notices.
Honestly I would never allow Brew on any company machines I have authority over. It is giving hundreds of randos, (and anyone that takes advantage of their poor opsec) the ability to execute any code on user systems.
Major Linux package managers do not go nearly far enough with things like review signing, but most at -least- do author-level package signing, human review, and independent reproduction for most packages.
Given how many high value targets like corporate sysadmins allow brew on their computers, Brew is on track to overshadow Crowdstrike any day now for most harm caused by insufficient supply chain management.
If macs supported PKGBUILDs like Pacman with a similar level of performance and there were correctly maintained packages for core programs I'd feel like using a mac would have a lot less compromises for convenience.
Homebrew is great and the formulae are maintained really well but the simplicity of PKGBUILDS, the fast syncing, and lack of cognitive burden of recalling multiple arguments/flags for package managers make me wish pacman just worked on macs.
The main attack vector IMHO is the simple fact that one can sneak in new packages with malicious intent by simply contributing a new formula.
The team of maintainers is too small to audit all of the newly contributed formulae.
I'm suprised that this attack vector wasn't part of the audit.
I don’t think the current Homebrew core formulae reviewers consider their team too small to sufficiently review all new incoming formula requests. But even if it was: this is one of the vagaries of packaging that’s explicitly called out in the post: the boundary between first- and third-party execution is inherently murky, and there’s IMO relatively more security “value” in determining where third-party execution can surprisingly happen than pointing out all of the unsurprising things that happen when you intentionally run third-party code.
(With that being said, I think packagaging ecosystems in general should be reviewed for those kinds of acceptance processes. But that would be closer to a “red team” style audit than a software audit, since it’s about human processes.)
Yeah, I just had a scare the other day with someone downloading a console emulator called "Cmder" which is a collection of a bunch of FOSS tools. It literally had ~1,000 files that could be malicious including powershell scripts, perl scripts, python scripts, shell scripts, DLLs, EXEs, etc. It turned out it was benign, but it's really scary that people just clone these Git repos and hope for the best.
Yep. We use the TOB-$PRODUCT-$XXXX convention for our audit findings, where $PRODUCT is the target under audit and $XXXX is a unique incrementing counter for each finding.
(As far as I know, a lot of audit firms do similar things.)
I’m a bit puzzled by the wording of this blog post, because it says you’ve worked with Homebrew to do this audit, but your name sounds familiar to me, and indeed if we check Homebrew’s README [1]:
> Homebrew's maintainers are […long list of names…] William Woodruff […]
I wasn’t a maintainer at the time I did the audit :-). I’ve been a non-maintaining “member” of the project for a long time, which is the pseudo-emeritus position we give to previous maintainers who want to continue participating in internal conversations and governance. I was then offered membership again, months after the audit, due to some unrelated work on Homebrew that didn’t exist and wasn’t planned before the audit was planned.
This was all disclosed as part of a conflict-of-interest disclosure I did, both with my company and with the Homebrew maintainers, but I agree that the blog post could also say that explicitly. I’ll try and get it added today.
TL;DR: I was not a maintainer at the time the audit was performed, but I was previously (years before) and am currently a maintainer. The audit was performed by myself and my colleagues in our professional capacities.
Kinda off topic, but when I see a project or an article use brew on Linux instead of their native package manager or something like flatpak I generally assume the author has very limited knowledge about Linux and I can ignore this project/article.
Even though Homebrew isn’t meant as a replacement for the system package manager, there are legitimate use cases.
For example, if you’re on Debian but need a newer version of just one tool, it makes absolute sense to install Homebrew alongside. It’s designed to play well with the system package manager, uses its own separate prefix so it won’t cause shared libs confusion, and only shadows the packages you install with it. Nothing wrong with that inherently, and certainly not a sign of limited knowledge.
Great that there are people looking into this. I wonder if there would be similar findings were they to perform an audit on MacPorts or the Nix package manager.
A while back I was trying to understand why Homebrew requires pre-built executables to be installed into /home/linuxbrew. I asked about it here[0]. This requirement basically makes it impossible to use homebrew to quickly install programs on systems where you don't have root, or at least have homebrew already configured (not sure if that would solve it but I assume so).
They pointed me to an example program that would break if not run this way: Facebook's Watchman[1].
It bizarrely (to me) has hard coded paths compiled into it, which force you to run it from specific directories.
Would love to understand what's going on here and why you would ever make software work this way. I feel I'm missing a fairly obvious Chesterton's Fence.
The short (but possibly not satisfying) answer is that Homebrew's relocation of packages (including binary relocation) is best effort, in part because of the myriad ways in which packages can embed absolute (or incorrect relative) paths and other state in their build products. macOS bottles are generally more relocatable (in part because of a lot of scar tissue around binary relocation), but it's a general problem with build system quality, build complexities, and - reasonably - disinterested upstreams.
In the case of Watchman, I have to assume that internal use is the most supported use case, and uniformity of deployment is desirable across the fleet there, and so, configurability wasn't as big of a concern?
This a not a unique problem to homebrew. Any pre-built binaries potentially shares this problem unless the build system the software use is intentionally written to avoid this problems.
Any package managers that is designed to not hard code the prefix, ie you can choose where the binaries go into, needs to handle this problem and have their own ways to deal with it. Conda for example has a long string of …placehold_placehold_… to facilitate editing the hardcoded path…
Source distribution is more robust against this problem comparing to binary distribution. (But sometimes the authors of the software did not package them well and would have hardcoded some paths somewhere.)
That’s why when you change the homebrew prefix, they will built from source instead, and it (using a different prefix) might not work.
The amount of value returned to the Apple ecosystem through brew is remarkable and while this post makes me even more in awe of the care that goes towards the community, I'm sad that one of the richest companies in the world isn't giving more back.
[+] [-] woodruffw|1 year ago|reply
If you're having trouble finding the audit itself (it's linked indirectly), I'm linking a copy here as well[1].
[1]: https://github.com/trailofbits/publications/blob/eb9344f2261...
[+] [-] greggsy|1 year ago|reply
I know it’s not the focus of a code review like this, but I’m interested to hear your views on the general supply chain lifecycle problems inherent to open-source package management platforms. Principally, are vetting processes appropriate to ensure that new formulas refer to the correct source? How does the user gain confidence that their brew update is still referencing a trusted source? What happens when a domain is taken over? How quickly can the team respond to untrusted sources from formulas?
I know these aren’t all Homebrew problems to solve, but they’re important ecosystem considerations.
(These problems also exist in the winget and choco platforms, but less so in commercially supported repos like apt and yum. For me, and many other admins, they are a major concern when it comes to the Windows Store.)
Edit: lastly, in case the homebrew team are watching: an npm-style vulnerability notice would be awesome
[+] [-] dotBen|1 year ago|reply
As a SWE/EngMgr turned VC, I'm curious if there's startups or commercial companies providing some kind of assurance here (but also worried the $ TAM for solving this problem probably isn't enough to make it a standalone business).
[+] [-] koito17|1 year ago|reply
In any case, I'd be interested in seeing an audit of Nix on Mac OS. Especially if there is a flaw in how `nix develop` and related commands work.
[+] [-] justusw|1 year ago|reply
Nix allows me to override most of that, and I can share home manager config with my Debian workstation.
[+] [-] tadfisher|1 year ago|reply
The Nix sandbox is also not really meant as a security boundary; there's no effort put into preventing sandbox escapes, and lots of stuff leaks from the host into the sandbox environment. You really want something like gVisor or a full VM if you want to build untrusted packages.
[+] [-] ninetyninenine|1 year ago|reply
[+] [-] saagarjha|1 year ago|reply
[+] [-] js2|1 year ago|reply
Even if I wrote this code and knew what it did today, if I come across this it a year from now, I'm left scratching my head: why did I make this change?
A real world example of a good commit message:
https://github.com/git/git/commit/92fe7c7d42cc941ed70d6fce98...
[+] [-] frenchman99|1 year ago|reply
[+] [-] pmarreck|1 year ago|reply
[+] [-] bokchoi|1 year ago|reply
[+] [-] pyjamafish|1 year ago|reply
I added a mention of ixnay to its readme :)
[+] [-] lrvick|1 year ago|reply
Homebrew maintainers mostly do not sign commits/packages, do not sign reviews/merges, do not verify author/reviewer sigs at compile time, do not reproduce builds in separately controlled CI, do not enforce hardware 2FA on Github.
Every user of brew is only as secure as whichever of hundreds of brew maintainers has the worst opsec today.
Also since dependabot automatically makes commits, you could get a malicious commit into an external project you control, wait for dependabot to make a commit to homebrew to upgrade it, then merge it yourself (as becoming a homebrew maintainer has almost no vetting, just fix a few easy bugs)
You could also just take over one of the expired email domains of a maintainer and send a password reset email to yourself and take over an account of someone on vacation or hiatus.
Can likely get thousands of companies compromised before anyone notices.
Honestly I would never allow Brew on any company machines I have authority over. It is giving hundreds of randos, (and anyone that takes advantage of their poor opsec) the ability to execute any code on user systems.
Major Linux package managers do not go nearly far enough with things like review signing, but most at -least- do author-level package signing, human review, and independent reproduction for most packages.
Given how many high value targets like corporate sysadmins allow brew on their computers, Brew is on track to overshadow Crowdstrike any day now for most harm caused by insufficient supply chain management.
[+] [-] arandomhuman|1 year ago|reply
Homebrew is great and the formulae are maintained really well but the simplicity of PKGBUILDS, the fast syncing, and lack of cognitive burden of recalling multiple arguments/flags for package managers make me wish pacman just worked on macs.
[+] [-] izik|1 year ago|reply
[1] https://github.com/liudongmiao/pacman [2] https://github.com/kladd/pacman-osx
[+] [-] pxc|1 year ago|reply
[+] [-] megamix|1 year ago|reply
[+] [-] tucosan|1 year ago|reply
[+] [-] woodruffw|1 year ago|reply
(With that being said, I think packagaging ecosystems in general should be reviewed for those kinds of acceptance processes. But that would be closer to a “red team” style audit than a software audit, since it’s about human processes.)
[+] [-] j16sdiz|1 year ago|reply
> ... These avenues do not necessarily violate Homebrew’s core security assumptions (which assume trustworthy formulae),...
[+] [-] Bluecobra|1 year ago|reply
[+] [-] yjftsjthsd-h|1 year ago|reply
Edit: Oh, it's "Trail Of Bits - homeBREW". But probably still yes.
[+] [-] woodruffw|1 year ago|reply
(As far as I know, a lot of audit firms do similar things.)
[+] [-] hk__2|1 year ago|reply
> Homebrew's maintainers are […long list of names…] William Woodruff […]
[1]: https://github.com/Homebrew/brew
Is there any reason this is not mentioned in the blog post? I don’t think it would make a difference, but just to clarify things.
[+] [-] woodruffw|1 year ago|reply
This was all disclosed as part of a conflict-of-interest disclosure I did, both with my company and with the Homebrew maintainers, but I agree that the blog post could also say that explicitly. I’ll try and get it added today.
TL;DR: I was not a maintainer at the time the audit was performed, but I was previously (years before) and am currently a maintainer. The audit was performed by myself and my colleagues in our professional capacities.
[+] [-] scovetta|1 year ago|reply
[+] [-] mootoday|1 year ago|reply
It abstracts Nix in a way you don't have to know or learn anything about the Nix language.
I wrote a few words on how I use it instead of Homebrew [1].
[1] https://mootoday.com/blog/i-replaced-homebrew-with-devbox
https://mootoday.com/blog/i-replaced-homebrew-with-devbox
[+] [-] charlie0|1 year ago|reply
[+] [-] daghamm|1 year ago|reply
[+] [-] Hackbraten|1 year ago|reply
For example, if you’re on Debian but need a newer version of just one tool, it makes absolute sense to install Homebrew alongside. It’s designed to play well with the system package manager, uses its own separate prefix so it won’t cause shared libs confusion, and only shadows the packages you install with it. Nothing wrong with that inherently, and certainly not a sign of limited knowledge.
[+] [-] efitz|1 year ago|reply
[+] [-] zargon|1 year ago|reply
[+] [-] eddyg|1 year ago|reply
[0] https://workbrew.com/
[+] [-] mikemcquaid|1 year ago|reply
[+] [-] amai|1 year ago|reply
[+] [-] unknown|1 year ago|reply
[deleted]
[+] [-] sohrob|1 year ago|reply
[+] [-] apitman|1 year ago|reply
They pointed me to an example program that would break if not run this way: Facebook's Watchman[1].
It bizarrely (to me) has hard coded paths compiled into it, which force you to run it from specific directories.
Would love to understand what's going on here and why you would ever make software work this way. I feel I'm missing a fairly obvious Chesterton's Fence.
[0]: https://github.com/orgs/Homebrew/discussions/5371
[1]: https://facebook.github.io/watchman/docs/install#prebuilt-bi...
[+] [-] woodruffw|1 year ago|reply
[+] [-] bagels|1 year ago|reply
[+] [-] KolenCh|1 year ago|reply
Any package managers that is designed to not hard code the prefix, ie you can choose where the binaries go into, needs to handle this problem and have their own ways to deal with it. Conda for example has a long string of …placehold_placehold_… to facilitate editing the hardcoded path…
Source distribution is more robust against this problem comparing to binary distribution. (But sometimes the authors of the software did not package them well and would have hardcoded some paths somewhere.)
That’s why when you change the homebrew prefix, they will built from source instead, and it (using a different prefix) might not work.
[+] [-] unknown|1 year ago|reply
[deleted]
[+] [-] razodactyl|1 year ago|reply
[+] [-] unknown|1 year ago|reply
[deleted]