top | item 33830915

The Audacity of Piping Curl to Bash

63 points| Un1corn | 3 years ago |yotam.net | reply

108 comments

order
[+] simiones|3 years ago|reply
This seems almost like a misunderstanding of what is the role of an installer, especially for something like oh my zsh. The author is complaining that it takes over their zsh configuration, when in fact that is obviously the whole point of the installer.

An installer isn't simply there to copy a program to your system. It's there to copy files to your system and then modify your system so that it is ready to use the new program to the deepest level that makes sense. You're not supposed to need to do any other configuration of your system for this program after the installer finishes in order to properly use it. This includes things like associating file types with this program, changing system settings to make it default in various places (hopefully with some kind of flag, to be fair), discovering and associating hardware or any other step like that.

Note that piping curl to bash or running bash on the output of curl/wget is a minor point quickly glossed over in the article, which is actually complaining much more about using custom installation scripts that do "too much".

[+] mjw1007|3 years ago|reply
The other part of an installer's job is to provide a reliable way to uninstall the program, without leaving any mess behind.

I think that's the main reason I'm reluctant to run curl|bash-ware. I might trust the authors not to be malicious, but I generally wouldn't trust them to be competent at cleaning up after themselves.

[+] revolvingocelot|3 years ago|reply
>An installer [... is] there to copy files to your system and then modify your system so that it is ready to use the new program to the deepest level that makes sense

Sure, but what's the corresponding step in the zsh "install"? Looks like copying over their ~/.zshrc. The "install" script could have chosen to clone the git repo, copy the file, then print "all done! run 'zsh' to start your new shell!' or whatever

[+] mustache_kimono|3 years ago|reply
> If my package manager had an Oh My Zsh package

This is the author missing the point. The reason `curl | bash` is common is because devs don't like packaging for every distro under the sun, and MacOS, and FreeBSD, and... If you really think `curl | bash` is the problem, then you should be lining up to package the stuff you use for your distro. Instead, it is always someone else's problem.

Package managers are great... for the user. For everyone else, a polyglot system, with arcane technical policies, and even more arcane human policies is... not ideal.

[+] ilyt|3 years ago|reply
The effort to package is precisely what the complaint is about; you have to tell (in package manifest) where your files land and which are app files (to be removed freely) and which are config (user might want to leave it, won't be automatically overwritten on update)

> devs don't like packaging for every distro under the sun

source .deb will generally works fine under Ubuntu, Debian and most flavours just fine unless you have some funky dependencies (and if you do, installer will also be complex)

RHEL/Centos RPM will cover near-whole rest of the market.

MacOS/FreeBSD will be different enough anyway that you will need to write a bunch more in install script

Building simple package that just delivers binary is not even that complicated. Getting them to pass distro muster is often harder but you don't need to do that.

> Package managers are great... for the user. For everyone else, a polyglot system, with arcane technical policies, and even more arcane human policies is... not ideal.

Most of those "arcane" policies are there so random incompetent dev won't fuck up other stuff in the system. Which is also why users want packages in the first place.

And you don't need to abide to any policy to make simple package that just puts your files on the system. Building dumb debian package is just few metadata files (post/preinst scripts + a file describing your package) in a directory and a single command.

You only need to worry about policies if you want to submit package to distro, and there are in place for a reason.

No shit they are better for users, that's their entire fucking point!

[+] bravetraveler|3 years ago|reply
With things like the OpenSUSE build system... I see this more as a one time cost

You write the spec files for the managers of choice; DEB, RPM, PKGBUILD, whatever

With that you parameterize the inputs. The version to build, where to get the sources, etc.

Maintaining these is... note your build/runtime requirements the same way you do while developing.

Once the specs are written the laborious work is finished. There are countless tools to make this less effort, eg: pyp2rpm and alien

I maintain packages through Fedora COPR, a similar system. These tools are my first pass at writing the spec for things I don't even own.

I practice what I'm preaching, and I really don't buy that it's a lot of effort. If you want users, do it.

This is a critical first step to being bundled in the distribution itself. You won't get maintainers if there's nothing to maintain.

[+] simiones|3 years ago|reply
It's even worse than that. The way common distros work is not "devs package software", it's "devs convince distro to pick up the package and maintain a fork".

Of course, you can run your own APT or RPM repo, but that actually asks your users for even more control of their systems (since you now have a way to get any new version more or less automatically installed on their system in perpetuity), and it makes it even harder for them to install your software.

You can also bundle your software as a .deb + .rpm + .[...] file instead of a .sh file, but there is really not that much difference.

[+] derefr|3 years ago|reply
> devs don't like packaging for every distro under the sun, and MacOS, and FreeBSD, and... If you really think `curl | bash` is the problem, then you should be lining up to package the stuff you use for your distro

I mean, no, those aren't the only two solutions. As with accessibility requirements for public-access shops, "we the users" could mandate (either through legislation, or more interestingly, through some kind of medical/legal-like software-engineering bar association) that devs either properly package their apps, or don't release them at all.

I'm not saying it's a good idea; just saying that it's probably the solution brewing in the back of the minds of people who write things like this.

[+] bombcar|3 years ago|reply
Even if omz had a package-managed package, it would do the basic curl|bash itself the moment it was installed, to update itself. Technically it's git and things, but that's all the curl|bash does anyway.
[+] Sakos|3 years ago|reply
> I would have just expected it to install it in the proper location (hopefully not in my home directory) and leave the rest of the configuration to me

While I don't advocate for piping curl to bash, this is exactly what I expect an installer to do. It should provide sane defaults that don't require me to fiddle around with manpages or other documentation and config files before I can even use the thing. I'd say that's even the standard for most software. Now, I might compromise on the installer telling me what command I need to enter to get a default configuration/setup integrated instead of doing it automatically, but I have too much shit to do to waste it on configuring the nth thing I've installed this week.

I think what's missing is some standardization around what an installer is allowed to do and flags to tell it when to make certain changes as well as explicit logging for what exactly was changed or added where, but that's not going to be solved if everybody has their own bespoke bash script for installation.

[+] spsesk117|3 years ago|reply
> I think what's missing is some standardization around what an installer is allowed to do

I don't mean to be facetious, genuinely curios, but to the authors point, isn't that the point of the package management system? It's a standardized and encapsulated way to provide software, with sane defaults, in an auditable way, that respects the users system.

I tend to agree with the author here, but I'm sympathetic to the maintainer: packaging for rpm/deb (in my experience) can sometimes be an enormous pain with many hoops to jump through, _especially_ if you're trying to get your package accepted upstream. With that said, it is a mature, standardized process, and is fairly painless in the self-hosted/non-upstreamed case.

[+] ilyt|3 years ago|reply
>I think what's missing is some standardization around what an installer is allowed to do and flags to tell it when to make certain changes as well as explicit logging for what exactly was changed or added where, but that's not going to be solved if everybody has their own bespoke bash script for installation.

YOU ARE RIGHT!

What's more, we can make common installer code so everyone could just use same rules and documentation to customize it to their liking.

And once we have many apps using it might even be integrated into system so you don't have to download as much, and all the bugfixes are in one place so we don't have thousand different install scripts

Once we have that we can just make a very simple data file format, say just a data.tar.gz for data and control.tar.gz for telling installer what to do. As for metadata, just write simple file, say

    Package: my-tools
    Version: 0.0.1
    Section: base
    Priority: optional
    Architecture: all
    Maintainer: Someuser <some@email>
If you need to run something after installer just put it in control.tar.gz as postinst script and it will do that

And if that's one format, we can it manage uninstalls too! Just put a simple text database of those files.

OH WAIT THAT'S A FUCKING DEBIAN PACKAGE MANAGER

[+] hprotagonist|3 years ago|reply

  curl ... | bash 
is the moral equivalent of

  {npm, pip, nuget, ...} install
and i really don't understand the folderol around that. In both cases, you can alter the command slightly to instead download the payload without executing it and inspect it first, if you wish. In both cases, you're ultimately going to either audit and then execute or just execute code from Somewhere Else.

This is true for distro package managers too, though you could argue that sometimes but not always (ppas, community/, whatever) a distro package manager is an extra layer of insulation between you and nasty stuff.

[+] Cyberdog|3 years ago|reply
This is kind of my take on it. As gross as I find pipe-to-shell installers to be instinctually, I can't really think of any objections I have about them which don't apply to just grabbing a package from MacPorts, save for one: MacPorts gives me a unified interface for listing and uninstalling that software after it's installed that I don't get from ad hoc installers. But in terms of the common complaints like security, it's pretty much the same - it's not like I'm auditing the source or patches of all the software I'm installing via MacPorts either.
[+] dllthomas|3 years ago|reply
If curl dies early, bash might be executing a truncated script. Arbitrarily truncated bash scripts are often valid bash scripts that do things you don't want.
[+] sjmulder|3 years ago|reply
Another downside of these scripts is that they tend to make changes to your user or machine configuration, something which is tolerated from Windows installers but a big no-no for me. E.g. I believe Cargo edits .profile to add its path and Teams makes itself start at login (!).

For reasons such as these, but also things like telemetry configuration defaults and clean uninstalling, I prefer using a package manager. In a way independent package maintainers balance out the power of upstream developers over end users. They embody “you can just change it if you don’t like it” for the regular user.

[+] steveklabnik|3 years ago|reply
Cargo does not. Rustup’s installer does, by default. It informs you of this before it does so, and you can ask it to not if you’d prefer.
[+] jerf|3 years ago|reply
Agree with author. For me, in addition to separating the "download script" and "run it" steps, there is also a "read script to figure out just what the heck it is going to do to my system", with optional "edit script to remove silly things" and "just manually run the three important commands" steps.

Unfortunately, there just isn't a way to square the circle of "this has to work for everybody" and " this shouldn't take 300 lines to ensure a directory exists".

[+] gorjusborg|3 years ago|reply
I post this every time this topic comes up.

Piping from the internet into your shell is a bad idea.

https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...

[+] rich_sasha|3 years ago|reply
I used to think that.

But is it really that much worse than `pip install` random stuff? Or Homebrew, or Linux package managers - it's not like these things get audited.

[+] lamontcg|3 years ago|reply
That is kind of nonsense. If someone already has hacked or MITM'd the server offering the install script payload the game is already up. No attacker is going to bother doing that, since the number of people who read the install script before sending it off to bash is minimal.
[+] Analemma_|3 years ago|reply
And every time you post it, a bunch of people point out that there is nothing going on here which package managers or installation executables can't also do.
[+] umvi|3 years ago|reply
> For the love of god, why do I still have programs on Linux that don’t use xdg directories?

Because a lot of devs have never heard of it? I'm a linux app dev of <10 years and I've never heard of xdg until this post. I just assumed dotfiles in the home directory were still the de facto standard...

[+] superkuh|3 years ago|reply
Not just applications. Entire programming languages like Rust depend on this to install the compiler (rustup, since rustc changes too fast for anything but the most rolling of distros to keep up).
[+] mustache_kimono|3 years ago|reply
> Entire programming languages like Rust depend on this to install the compiler

And it works! For people who have to stay on top of language changes (not everyone), who are on a wonky platform or that doesn't update quickly enough, this is actually a pretty okay method. And they do also offer alternative methods: https://rust-lang.github.io/rustup/installation/other.html

Here, the "You know what they should be doing..." attitude surely doesn't account for something.

> rustc changes too fast for anything but the most rolling of distros to keep up

This may have once been true, but isn't really anymore. Ubuntu updates all distros with the latest and greatest and does so about every couple of minor version releases (last was 1.59 to 1.61). This is fine for me. My MSRV is now whatever Ubuntu is shipping.

[+] rascul|3 years ago|reply
It's one way out of several to install rust. And interestingly enough, my distro maintains a more up to date rust than I do myself.
[+] geocrasher|3 years ago|reply
My issue with piping curl to bash is that so many of these installers are pure junk.

Case in point: I work in web hosting. Yesterday a customer came to me asking for root access to the node so they could run an installer for something. No. But they had already tried running it as their user. And everything in their user account was gone. Why?

Because the installer expected to run as root, and its variables couldn't be defined properly and so when it went to clean up after itself, it did

   rm -rf ~/$variable/
and since the variable was unassigned, that became

   rm -rf ~/
I might not have it exactly right, but that's what the effect was. Piping curl to bash is asking a lot of somebody who doesn't know what they're doing, and should raise the hackles of somebody who does. At the very least, download and view the script yourself before running it.
[+] Joker_vD|3 years ago|reply
> why do I still have programs on Linux that don’t use xdg directories?

Because there are Linux developers who never heard of XDG and just put their stuff wherever. And since ignoring XDG doesn't makes your application completely unusable, they have pretty much zero incentive to learn about it. Crazy world, isn't it?

[+] ragingrobot|3 years ago|reply
Quite a few applications do this.

For something like the mentioned oh-my-zsh, it can be safely assumed the user is not a novice in most cases. Having to install in this manner may in fact deter the user, as they'd be suspicious. A well written README would be the better route.

[+] ArchD|3 years ago|reply
When I want to be careful about running these setup scripts, especially just for trying out new software, I run them in a docker container to limit whatever damage the scripts can cause. When the script is complicated, I tend to use docker instead of trying to understand the script and then run it on the 'real' system.

Then when I really like the software and want to install it on the 'real' system, if there's much benefit in doing so, I spend more time and effort understanding the script. More often than not, I end up not doing this because there is no compelling need to install the software on the 'real' system.

[+] manv1|3 years ago|reply
Really, the risk here is that the install is going to do something unfortunate, like delete everything in your filesystem because you have a space in your home directory name or cause problems because your .profile didn't end with a CR and it blindly appended it's own stuff to it.

I'm not sure how package managers prevent this sort of issue, but in general running shell scripts as root (and it probably needs to run as root) is a bad thing.

[+] halayli|3 years ago|reply
The title and the content don't match. The omzsh installer behavior is orthogonal to piping it from curl to bash.

On the other hand, It's often the case that your machine is running scripts that have been fetched online via apt and the like and it's definitely something to consider especially with all the hacks that have been happening in the last few years and the undisclosed vulnerabilities available in the wild.

[+] Brian_K_White|3 years ago|reply
Reading all the comments not understaning the problem is a great way to feel old. It's definitely a new generation, only in the bad way where instead of meaning new energy, imagination, and progrrss, it just means forgot or never learned important concepts and principles.

You do not take liberties with someone else's system, there is no need to do it and no excuse for it. You can have a reference example "make install" in your build system that serves as a reference for the packagers without you having to worry about all the 80 different distros. And it better also have a "make uninstall".

Respecting the possibility that a config file or even the bins and libs might already exist as part of the "make install", are just part of the job like writing the software itself, not some unreasonable extra burden.

If you're that much of a baby then I do not want your 'free' gift software and nor should anyone else. What other corners are you cutting everywhere else in the software? What other gross lack of integrity do you think is ok?

Maybe this is more the result of turning every random application into it's own cpntainer. It's fine to have an app installer configure the entire system to suit itself when the entire system is just the container to house the app.

[+] simiones|3 years ago|reply
> You do not take liberties with someone else's system, there is no need to do it and no excuse for it.

The whole point of the oh-my-zsh installation script is to modify your system to work with oh-my-zsh. If you don't want your system modified, you shouldn't run it: there is no other point of that script.

Build instructions are a completely separate thing, and are a complete distraction. No one sane waits for some random distro to discover your software and decide to package it themselves as a means of distributing it.

As far as most people are concerned, the role of things like apt or rpm is to manage the base system. Installing and keeping application software up to date is best left to the applications themselves - as it has always been on Windows or MacOS (before the app store craze), as it should be. It is not and should not be up to the Debian maintainers to tell me what version of Firefox to use, or how often I should update it.

Edit:

> Respecting the possibility that a config file or even the bins and libs might already exist as part of the "make install", are just part of the job like writing the software itself, not some unreasonable extra burden.

I assume you are referring to the author's complaint about the installer overriding their ~/.zshrc. If so, then that is again a misunderstanding of the point of this script - it explicitly tells you right in the description that it will do that AND it keeps the old file around in case you still need it.

To explain again - oh-my-zsh is a system for controlling your zsh installation. It's whole purpose is to take over things like your .zhsrc file. This is explained very clearly on their main page, so running that script and expecting it to not modify your zsh settings is like installing Firefox and expecting it not to connect to the Internet when you type a URL in the address bar.

[+] NuSkooler|3 years ago|reply
Package managers are sometimes great for the user, and as mentioned, a pain for the developer in many cases. To cover even "basic" bases, the developer has to manage many package managers. Ouch.

From the user side, the package manager often doesn't do what I want, either. I could install Node (as an example) via `apt` or `yum`, and end up with a Node installed in a root location. Now I'm in a mess. Or I could use a install script, or even yet another 3rd party solution such as `npm` to do what I actually want: Node installed for me. ...of course, I just mentioned a whole other can of worms: All the "other" package managers out there.

TL;DR: KISS often is the best solution.

[+] uuddlrlrbaba|3 years ago|reply
Dive into the details of what "clean" packaging entails, how much practices differs between distributions, how many distributions there are, how each dependency also needs to be packaged and maintained across upgrades...

And you'll quickly see why projects say fsck it -- we support installation via curl | bash. go and package it yourself it you want to.

It really highlights the need for a broadly adopted "homebrew for linux" type package manager that could safely manage software without conflicting with OS packages.

[+] derefr|3 years ago|reply
Homebrew is Homebrew for Linux. You can install and use brew(1) on Linux. The Homebrew github worker even generates linux/amd64 binary "bottles" for each Homebrew package to make it fast (as long as the package doesn't explicitly opt out of Linux support.)
[+] mdaniel|3 years ago|reply
> "homebrew for linux" type package manager that could safely manage software without conflicting with OS packages.

As someone who uses Homebrew for Linux, I can say that "without conflicting with OS packages" cuts both ways: fine, I get more modern stuff than apt could imagine, but having to monkey with the LD_LIBRARY_PATH or -Wl,-rpath over and over gets old real fast. I have no idea why they tried to be so cute putting things in a stupid directory (/home/linuxbrew/.linuxbrew) instead of /usr/local like they did with Homebrew for Mac (err, not the arm64 version, where they went back to /opt/homebrew for whoknowswhy)

[+] nimbius|3 years ago|reply
piping curl to bash is a heresy, an abomination that cannot be tolerated. it is an affront to my intellect and sensibility. it is an ethical turpitude and degeneracy that cannot be understated in its depravity.

It reduces the user to nothing more than an endured, pseudotrutworthy ball of lard in the developers formal equation of installation. As it is an installation however it beseeches the administrator, the root, the owner and the light of this system that it may achieve its purpose and in doing so is a blasphemy. it supplants my GNUlike will and in its stead enforces the hopes and dreams of nothing more than a transient, a visitor.

For the last time: The ground your code touches is holy and hallowed. the rites of Posix and the decree of the Unix philosophy at the sides of the throne you approach alone implores you not to speak unless spoken to, unless absolutely in the favor of the god of this land. To sudo curl|/bin/bash is to commit an unspeakable treason in the divine presence, a sin unforgiveable before the light of the PTY and the TTY. To take the sudo sword of the emperor alone and wield it as you see fit is damnable contempt indeed.

[+] daitangio|3 years ago|reply
bash installer are neat, fast and dirty. They shines inside a Dockerfile, because enable to install not-yet-packaged software.

To avoid regret their launch, I enabled sudo to ask me a password, to avoid some sudo malicious command in the wild destroying my box or wiping my nas drives...who knows? :)