I feel like screaming, who thought this was a good idea? If I invoke something as sudo, why does anyone think it should try to detect that and do anything about it? I want to run as the user sudo has set, not my own user, OBVIOUSLY.
Don't try to be smart about sudo, you will break stuff.
Of course. Try running npm on a shared fs (fusehgfs). Most things don't work. This is just shit software and the people who make it need to concern themselves with fixing their software rather than trying to prevent imaginary security problems they shouldn't be trying to prevent anyway (I don't know what their security delusions are exactly, but I do know they are delusions). I'm an engineer and a sysadmin and I I know when something should be run as root or not. That should be my choice and my choice alone. I shouldn't be nagged about it and I certainly shouldn't be prevented. But this is npm. That's why people created yarn, I think.
But in fairness, I can't count the number of times that I've needed to fix things after people treated `sudo npm` as Simon Says[1].
I'm sure they struggled a lot with that issue before coming to this solution. Was it the right solution? Absolutely not. But that's not the point I'm trying to make.
It's all too easy to tunnel vision on a particular solution. I've done it plenty of times, and I'm thankful to those who have helped me to see other alternatives in time.
I'd go further and say that chown should be "considered harmful".
There are 3 use cases I can think for chown(2):
- Implementing the chown command (or other tools whose purpose is explicitly and only to manage permissions)
- Implementing a file copy/archive command that preserves permissions
- For package managers that set up a daemon user for a package, and want to set up the a writable area of the fs for use by that user
In other words, the ownership of files is something that should be totally up to the user, and not something implicitly done by a tool on their behalf.
I can't think of a single other place where trying to automatically manage file ownership is warranted. Files I touch should be owned by me, files root touches should be owned by root, and the correct way to make sure new files are not owned by root is to not be root. Doing literally anything else with chown is being overly clever and is a guaranteed landmine.
The idea that correctMkdir() exists at all seems to me to be so wrong-headed.
This comment from the source says a lot:
// annoying humans and their expectations!
Good UX is an important, oft-overlooked consideration, but there is definitely such a thing as taking it too far. If your humans are expecting this level of hand-holding, it's because you've trained them to expect it by pandering to them up until now. This is the kind of problem that should be handled with good, detailed, error message display when users don't get the result they expect, not "fixing" it with over-reaching magic.
I'm not sure I'd trust anything put out by the npm team in general from hereonin if they genuinely thought creating the correct-mkdir.js file in the first place was a reasonable idea. Is it? Genuinely open to a counter-argument.
It all started with ubuntu pointing home to the user's dir when running sudo. This was done out of convenience to have gedit and other Xorg apps work when run with sudo...
Then there is also the terrible fact that ~/.local/bin didn't exist as a "standard" at the time. Which means your only sure-fire non-complicated way to install local bins guaranteed to work for the user was to put them in /usr/local/bin which meant running sudo.
But if you create a package cache dir during sudo in ubuntu in $HOME, thats with root permissions! Then you get errors when trying to run npm without root and it tries to manipulate the cache. How do we fix this, by changing cache dir permissions of course. https://github.com/npm/npm/commit/ebd0b32510f48f5773b9dd2e36...
Through series of refactorings, this became correctMkdirp, which is a non descriptive name of mkdir with (among other things) changing of permissions. And with a name like that it eventually was used in the wrong context and did the wrong thing.
I call this death-by-10-small-missteps. But I would pin the biggest problem on a missing omnipresent `~/.local/bin` standard (at the time). It doesn't cost much if anything, and it would single-handedly obliterate the need for users to much around in their paths (bad usability) or run sudo to install command line tools for personal use (clearly not the best idea).
I initially parsed that comment as "this library exists to annoy humans" rather than "we wrote this to satisfy humans, who are annoying in this respect".
There appear to be no unit tests for their entire lib/utils folder. Which includes things like this (misguided) chown utility. https://github.com/npm/npm/tree/release-next/test - and note the lack of testing in the commit linked in the bug report.
I had an inkling that NPM was cancer, but not like this.
It's been almost 2 years since the great left-pad debacle[0]. The last major npm issue[1] was less than 2 months ago. While the underlying npm registry security issues will remain for a while (and other languages don't seem to have these issues with their package managers), there doesn't seem like there's too much I can do other than use yarn. And hope an alternative registry will appear.
It took me ~5 mins to migrate all of my code from npm to yarn. But I don't have complex CI tasks either.
I use ncu to check updates every couple of days, sometimes more frequently. To further distance myself from npm, can anyone comment on the pros/cons of github repo paths instead of package names in package.json?
Github paths can change way more easily than npm packages, users can rewrite git history + break your stuff and versioning when using it with npm is horrible.
NPM also now protects projects from namesquatting and prevents you from deleting them when multiple projects depend on them.
The left-pad debacle was a registry issue but the current issue is an npm client issue. In my experience, NPM 5 client versions have been shaky and unreliable, and there are problems with popular ecosystems like react-native [0]. I always roll back to npm 4, even on node 9.
As far as I know it's not feasible to use Github repos instead of npm, at least for client-side code. Most packages are compiled from ES6 (or typescript, jsx etc) to ES5, and the generated code (which is what you want in your project) is usually not included in the git repo.
I just can't feel sorry for folks when I see comments like this one:
> This destroyed 3 production server after a single deploy!
I do think that the developers have a duty to do some testing of their software before putting out releases/updates. However, users also have a duty to perform sufficient testing before they push new versions to their production environments.
In my opinion, it's kinda like losing data because you didn't make and/or test your backups. It's a really crappy way to have to learn a lesson but at least they've finally learned it -- and if they haven't, well, then maybe they will the next time it happens.
What you say is 100% true, but I would go one step further and not have npm installed on any production server.
And it's not specific to npm, I would do the same with gem, pip, cpan, etc. Not to mention curl http://ex.io/install.sh | sudo bash.
Call me old school, but personally, I would avoid installing anything from language specific package managers. I would instead either build an rpm/deb package for every dependencies as a single package or if it's too complex, bundle the dependencies and the application in one package which deploys the bundle under /opt/.
That way I only have one source to check in order to see what is installed on my systems. Also, rpm and dpkg tends to be far better at managing what is installed by each package, and far better at uninstalling everything during cleanups.
Also, mixing a language specific package manager and a distribution package manager can have unforeseen side effects as the two can step on one another (for example, I ran into issues recently with a pip install python-consul overlapping with a yum install salt-minion as both of them download python-requests as a dependency).
I am the one who reported this ;) In fact that was a single production server that I tried to reinstall 3 times before catching it was not really one of the commits that was doing this. No data was lost or connectivity (as long as you do not reboot it), you just lose any ssh connection/login.
Should I have done this on a staging server? Sure, but that does not change the fact that I would have had to rebuild the whole server there too. It is not expected that updating npm will kill the complete system it is on... It would be expected to have some deploy failure of some sort.
As previously noted, `npm update -g npm` pulls in version 5.7.0. Version 5.6 is still the latest but for some obscure reason if you have thisupdate anywhere in your deploy script you are screwed.
If he'd tested it beforehand the comment might instead be "this destroyed my laptop".
You have a responsibility to test before releasing to production, yes. But the amount of fucked up your program has to be for `sudo ___ --help` to wreck the operating system, the unexpectedness of that result... IMO attention should be focused here on the irresponsibility of the npm team, not their users.
To play devil's advocate, this was a dev version, without any UA or E2E testing, tagged as a major release. This after there have been major bugs for several releases. This isn't 0.7.0.
Perhaps the problem is that the stability of the world's fastest growing development platform hangs on the implementation of best practices by a two person developer team.
NPM needs to step up its game or we need to make something like yarn the standard.
Good lord, when I try to follow the link I get the Unicorn error page with the message 'This page is taking way too long to load. Sorry about that. Please try refreshing and contact us if the problem persists.'
Has this issue provoked so much outrage that GitHub can't handle the constant stream of angry emojis on the issue comment thread?
Apart from being a horrific bug, why are people running npm as root? Why don't they install it somewhere below $HOME and modify $PATH? npm is working fine without root permissions.
Everything is super dangerous as root, one should avoid using root at all costs until there is no other way.
`sudo npm install -g` is one of several examples of the normalization of deviance rife in the NodeJS community. Most command-line utilities distributed through NPM recommend running as root (implicitly—because they all suggest installing it as a global package). Here's[1] Microsoft's instructions to install the TypeScript compiler, for example.
NPM's awfulness notwithstanding, it's trivial to write a shell script to do what you say and add a symlink to ~/bin. But everyone on StackOverflow will tell each other "just run it with sudo", and they do, and then quickly move on with their lives (presumably to be followed with "and break things"). Instead of doing the right thing, raising their hackles about how poorly NPM is designed, and holding its community leaders accountable.
I know a lot of inexperienced devs who run things as root to try to fix problems. It’s the first thing some of them do (after restarting). I think it’s a lack of awareness due to missing knowledge. When you don’t know much about how your system works, you can’t effectively troubleshoot it.
Doesn't really surprise me when you have other issues like this (https://github.com/npm/npm/issues/17929) that have persisted for a long time. NPM 5.x in general hasn't been very stable.
Indeed. Another odd thing that it's been doing lately is when I run some NPM scripts on one of our machines, it starts shouting about some sort of update not working (why was it updating anything at all just because I ran `npm run something`?) and gives me instructions on how to fix it from the Linux shell (on a Windows box). The depth of failure implied by that message is disturbing on several levels.
Wow the toxicity on that thread is appalling. I feel like I need to a tool when hiring people that automatically shows me their github comments with the most reactions.
"Thankfully, it only affected users running npm@next, which is part of our staggered release system, which we use to prevent issues like this from going out into the wider world before we can catch them. Users on latest would have never seen this!"
If you are updating to the latest pre-release of something within mere hours of it dropping and you are updating production systems (presumably that have some business value) with no previous testing then the consequences of that aren't on the devs they are 100% on you. And you don't deserve to call yourself an IT (or Ops or DevOps or what-have-you) professional, that is amateurish behavior in the extreme.
My personal opinion is that the root cause of the issue is the ability of a language pacakge manager to mess with system files at all (i.e. do a global install of anything). Shards, the crystal package manager makes the sensible design decision to only install libraries into `$PWD/lib` and binaries into `$PWD/bin`. Everything is local only to your project. If you want a binary on your PATH, you can create an installation method that works for your commandline tool's specific usecase. Hopefully a distro/homebrew package.
npm is one of the few tools that I am afraid to have on my Laptop, Because unlike most tools I have used, When npm does something wrong, It'll ruin not just itself but a lot more directories on my pc which is annoying to fix.
Oh well, I remember fondly that one time I had an important deadline whooshing by (with that lovely sound Douglas Adams knew) and I happened across this cute little bug:
I do remember scrambling to recover my backups. Back then, I didn't make full-disk backups, so I had to assemble my user folder from various places. Everything else that transpired that night and the day after remains a haze.
I find it interesting that nobody noticed this before public release. And apparently this version is a pre-release? But that isn't specified on the blog post?
As a semi-outsider to the frontend and node development worlds, it continues to surprise me that a viable alternative to npm still hasn't come along. Not trying to pile more hate on npm, but there's been many years of complaints about instability, horrid UX, bad security model, user hostility, etc. Yarn was just a first step. If there was a system with half the features, but made sense and was secure I think the community would shift very quickly.
[+] [-] zaarn|8 years ago|reply
Looks like the line responsible checks if the npm binary is run as sudo and then uses the UID and GID of the invoking user when chowning the directory. [ https://github.com/npm/npm/blob/latest/lib/utils/correct-mkd... ]
I feel like screaming, who thought this was a good idea? If I invoke something as sudo, why does anyone think it should try to detect that and do anything about it? I want to run as the user sudo has set, not my own user, OBVIOUSLY.
Don't try to be smart about sudo, you will break stuff.
[+] [-] SilasX|8 years ago|reply
[+] [-] mnm1|8 years ago|reply
[+] [-] reificator|8 years ago|reply
But in fairness, I can't count the number of times that I've needed to fix things after people treated `sudo npm` as Simon Says[1].
I'm sure they struggled a lot with that issue before coming to this solution. Was it the right solution? Absolutely not. But that's not the point I'm trying to make.
It's all too easy to tunnel vision on a particular solution. I've done it plenty of times, and I'm thankful to those who have helped me to see other alternatives in time.
[1]: https://xkcd.com/149/
[+] [-] tomc1985|8 years ago|reply
[+] [-] ninkendo|8 years ago|reply
There are 3 use cases I can think for chown(2):
- Implementing the chown command (or other tools whose purpose is explicitly and only to manage permissions)
- Implementing a file copy/archive command that preserves permissions
- For package managers that set up a daemon user for a package, and want to set up the a writable area of the fs for use by that user
In other words, the ownership of files is something that should be totally up to the user, and not something implicitly done by a tool on their behalf.
I can't think of a single other place where trying to automatically manage file ownership is warranted. Files I touch should be owned by me, files root touches should be owned by root, and the correct way to make sure new files are not owned by root is to not be root. Doing literally anything else with chown is being overly clever and is a guaranteed landmine.
[+] [-] lucideer|8 years ago|reply
The idea that correctMkdir() exists at all seems to me to be so wrong-headed.
This comment from the source says a lot:
Good UX is an important, oft-overlooked consideration, but there is definitely such a thing as taking it too far. If your humans are expecting this level of hand-holding, it's because you've trained them to expect it by pandering to them up until now. This is the kind of problem that should be handled with good, detailed, error message display when users don't get the result they expect, not "fixing" it with over-reaching magic.I'm not sure I'd trust anything put out by the npm team in general from hereonin if they genuinely thought creating the correct-mkdir.js file in the first place was a reasonable idea. Is it? Genuinely open to a counter-argument.
[+] [-] spion|8 years ago|reply
It all started with ubuntu pointing home to the user's dir when running sudo. This was done out of convenience to have gedit and other Xorg apps work when run with sudo...
Then there is also the terrible fact that ~/.local/bin didn't exist as a "standard" at the time. Which means your only sure-fire non-complicated way to install local bins guaranteed to work for the user was to put them in /usr/local/bin which meant running sudo.
But if you create a package cache dir during sudo in ubuntu in $HOME, thats with root permissions! Then you get errors when trying to run npm without root and it tries to manipulate the cache. How do we fix this, by changing cache dir permissions of course. https://github.com/npm/npm/commit/ebd0b32510f48f5773b9dd2e36...
Through series of refactorings, this became correctMkdirp, which is a non descriptive name of mkdir with (among other things) changing of permissions. And with a name like that it eventually was used in the wrong context and did the wrong thing.
I call this death-by-10-small-missteps. But I would pin the biggest problem on a missing omnipresent `~/.local/bin` standard (at the time). It doesn't cost much if anything, and it would single-handedly obliterate the need for users to much around in their paths (bad usability) or run sudo to install command line tools for personal use (clearly not the best idea).
[+] [-] jergason|8 years ago|reply
The correctMkdir change seems more recent, but not really related to that specific comment.
[+] [-] SilasX|8 years ago|reply
[+] [-] btown|8 years ago|reply
I had an inkling that NPM was cancer, but not like this.
Yarn, by contrast, has everything you would expect of a Facebook-engineered library: https://github.com/yarnpkg/yarn/tree/master/__tests__/util
Will be closely evaluating a switch to Yarn for our live apps. This is simply sad.
[+] [-] grahamj|8 years ago|reply
So it collects your personal information, even when not using it, and uses it for profit?
[+] [-] lhorie|8 years ago|reply
https://github.com/npm/npm/pull/19889
This kind of thing disintegrates my confidence on npm as a project.
[+] [-] officialchicken|8 years ago|reply
Since I 'vote' with my code - this migration page has been helpful today - and I hope it will help others: https://yarnpkg.com/lang/en/docs/migrating-from-npm/
It took me ~5 mins to migrate all of my code from npm to yarn. But I don't have complex CI tasks either.
I use ncu to check updates every couple of days, sometimes more frequently. To further distance myself from npm, can anyone comment on the pros/cons of github repo paths instead of package names in package.json?
[0] https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos
[1] https://github.com/npm/registry/issues/255
*edit: formatting
[+] [-] explodingcamera|8 years ago|reply
[+] [-] TAForObvReasons|8 years ago|reply
[0] https://github.com/facebook/react-native/issues/14209
[+] [-] JonRB|8 years ago|reply
[0] https://disjoint.ca/til/2017/11/10/managing-package-dependen...
[+] [-] po1nter|8 years ago|reply
You can use "yarn outdated" for that.
[+] [-] yogaboll|8 years ago|reply
[+] [-] jlgaddis|8 years ago|reply
> This destroyed 3 production server after a single deploy!
I do think that the developers have a duty to do some testing of their software before putting out releases/updates. However, users also have a duty to perform sufficient testing before they push new versions to their production environments.
In my opinion, it's kinda like losing data because you didn't make and/or test your backups. It's a really crappy way to have to learn a lesson but at least they've finally learned it -- and if they haven't, well, then maybe they will the next time it happens.
[+] [-] kakwa_|8 years ago|reply
And it's not specific to npm, I would do the same with gem, pip, cpan, etc. Not to mention curl http://ex.io/install.sh | sudo bash.
Call me old school, but personally, I would avoid installing anything from language specific package managers. I would instead either build an rpm/deb package for every dependencies as a single package or if it's too complex, bundle the dependencies and the application in one package which deploys the bundle under /opt/.
That way I only have one source to check in order to see what is installed on my systems. Also, rpm and dpkg tends to be far better at managing what is installed by each package, and far better at uninstalling everything during cleanups.
Also, mixing a language specific package manager and a distribution package manager can have unforeseen side effects as the two can step on one another (for example, I ran into issues recently with a pip install python-consul overlapping with a yum install salt-minion as both of them download python-requests as a dependency).
[+] [-] jguimont|8 years ago|reply
Should I have done this on a staging server? Sure, but that does not change the fact that I would have had to rebuild the whole server there too. It is not expected that updating npm will kill the complete system it is on... It would be expected to have some deploy failure of some sort.
As previously noted, `npm update -g npm` pulls in version 5.7.0. Version 5.6 is still the latest but for some obscure reason if you have thisupdate anywhere in your deploy script you are screwed.
[+] [-] rad88|8 years ago|reply
You have a responsibility to test before releasing to production, yes. But the amount of fucked up your program has to be for `sudo ___ --help` to wreck the operating system, the unexpectedness of that result... IMO attention should be focused here on the irresponsibility of the npm team, not their users.
[+] [-] fortythirteen|8 years ago|reply
Perhaps the problem is that the stability of the world's fastest growing development platform hangs on the implementation of best practices by a two person developer team.
NPM needs to step up its game or we need to make something like yarn the standard.
[+] [-] amelius|8 years ago|reply
[+] [-] zyztem|8 years ago|reply
Correction will be little harder on Debian derivatives and whole incident can be completely prevented on Solaris with file-mac-profile.
[+] [-] boffinism|8 years ago|reply
Has this issue provoked so much outrage that GitHub can't handle the constant stream of angry emojis on the issue comment thread?
[+] [-] kaiby|8 years ago|reply
[+] [-] fokinsean|8 years ago|reply
[+] [-] SilasX|8 years ago|reply
[+] [-] jlgaddis|8 years ago|reply
[+] [-] foepys|8 years ago|reply
Everything is super dangerous as root, one should avoid using root at all costs until there is no other way.
[+] [-] carussell|8 years ago|reply
NPM's awfulness notwithstanding, it's trivial to write a shell script to do what you say and add a symlink to ~/bin. But everyone on StackOverflow will tell each other "just run it with sudo", and they do, and then quickly move on with their lives (presumably to be followed with "and break things"). Instead of doing the right thing, raising their hackles about how poorly NPM is designed, and holding its community leaders accountable.
1. https://github.com/Microsoft/TypeScript/blob/b29e0c9e3ab2471...
[+] [-] tachyoff|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] master-litty|8 years ago|reply
[+] [-] mkj|8 years ago|reply
[+] [-] hysan|8 years ago|reply
[+] [-] Silhouette|8 years ago|reply
Indeed. Another odd thing that it's been doing lately is when I run some NPM scripts on one of our machines, it starts shouting about some sort of update not working (why was it updating anything at all just because I ran `npm run something`?) and gives me instructions on how to fix it from the Linux shell (on a Windows box). The depth of failure implied by that message is disturbing on several levels.
[+] [-] nwhatt|8 years ago|reply
[+] [-] akras14|8 years ago|reply
[+] [-] InclinedPlane|8 years ago|reply
"Thankfully, it only affected users running npm@next, which is part of our staggered release system, which we use to prevent issues like this from going out into the wider world before we can catch them. Users on latest would have never seen this!"
If you are updating to the latest pre-release of something within mere hours of it dropping and you are updating production systems (presumably that have some business value) with no previous testing then the consequences of that aren't on the devs they are 100% on you. And you don't deserve to call yourself an IT (or Ops or DevOps or what-have-you) professional, that is amateurish behavior in the extreme.
[+] [-] RX14|8 years ago|reply
I wrote about this in longer form here: https://github.com/crystal-lang/crystal/pull/3328#issuecomme....
[+] [-] ishanjain28|8 years ago|reply
[+] [-] dictum|8 years ago|reply
http://appleinsider.com/articles/09/10/12/snow_leopard_guest...
(Yeah, it's that much-vaunted Snow Leopard.)
I do remember scrambling to recover my backups. Back then, I didn't make full-disk backups, so I had to assemble my user folder from various places. Everything else that transpired that night and the day after remains a haze.
[+] [-] jehlakj|8 years ago|reply
[+] [-] igotsideas|8 years ago|reply
[+] [-] obw|8 years ago|reply
[+] [-] dlandis|8 years ago|reply
[+] [-] whatever_dude|8 years ago|reply
With something that has as many people using it, it's just... I dunno, it's disheartening.
Edit: oh well, this was a @next release only. Not as bad. Still scary.