The fact that this is possible with NPM seems really dangerous. The author unpublished (erm, "liberated") over 250 NPM modules, making those global names (e.g. "map", "alert", "iframe", "subscription", etc) available for anyone to register and replace with any code they wish.
Since these libs are now baked into various package.json configuration files (some with 10s of thousands of installs per month, "left-pad" with 2.5M/month), meaning a malicious actor could publish a new patch version bump (for every major and minor version combination) of these libs and ship whatever they want to future npm builds. Because most package.json configs use the "^1.0.1" caret convention (and npm --save defaults to this mode), the vast majority of future installs could grab the malicious version.
@seldo Is there a plan to address this? If I'm understanding this right, it seems pretty scary :|
So we need gpg signed packages :> And... all packages should be namespaced under the author who published them. And... I kind of want to say "once it's published, it's forever".
Do they seriously not use any sort of public key cryptography to sign packages? Frankly, for a project that's been around as long as NPM, that's downright irresponsible if that's the case. That's like package management 101.
I'm left thinking of how Go does it, whereby repositories are downloaded via git or whatever version control software's URL. Making it impossible for the admins of "NPM" to take down a package. Add in IPFS and you've got one heck of a "package manager" I guess Go's approach is not really a "package manager" but nobody can truly stop you from downloading from a git repository, and if they do, you can go to the forks etc.
Legally isn't it totally NPM as a company to publish existing versions of left-pad because those versions were already released under a license that allows redistribution, whether the author wants them to or not? Or can the author effectively veto this?
This is extremely severe. Any package i install might after x levels of sub-dependencies pull in one of these names which are potentially pwned. React and Babel pulled in a few of them to take some well known examples.
I would say the whole npm is pwned until these packages are either restored or that the package name is blacklisted/reserved. Is it possible to blacklist packages locally so I get warned if they are pulled in as sub dependencies? I don't trust every package maintainer to be quick enough to remove these dependencies.
Because NPM knows a module has been unpublished, it can try to be smart about it and change existing package.json so there will be no more updates to that module.
I also think dependencies should be non-flat/tree/static. So that you only have to trust one dependency, and not all of it's child dependencies and their dependencies too. You should only need to manage your own dependencies, not other peoples dependencies.
There should also be a way to contact everyone who use your module, so you can tell them about critical bugs etc.
One interesting thing to me, is that it is pretty clear that the kik lawyers pretty dramatically over enforced their trademark.
For those who don't know, the purpose of trademarks is to prevent customer confusion; essentially we don't want people to be able to sell cheap knock-offs of someone else's thing without the general public being able to easily distinguish between them. In practical terms, trademarks are "scoped" by their "goods and services" declarations.
For example, Apple the device manufacture[1] and Apple the record label[2] could both be trademarked because they had non-overlapping goods and services declarations... until iTunes started selling music[3].
If you look at kik's trademark application[4], you can clearly see that the trademark is limited to chat/media consumer applications, a pretty obvious over enforcement.
Most Germans wouldn't know that kik is also a chat app ;-)
"KiK is the largest textile discounter chain in Germany and operates about 3,200 stores in Germany, Austria (since 1998), Slovenia and Czech Republic (since 2007), Hungary and Slovakia (since 2008), Croatia (since 2011) and Poland (since March 2012)."
The Apple thing doesn't work though, for a specific reason: it's a very common word/name.
Smith is a very common surname. If one person starts the Smith Automobile Company and another person starts the Smith Farm, obviously there's no issue there. That's because none of us invented "Smith", it's understood to be a common name, etc.
On the other hand, if I start "The Google Paper Company", I'm pretty damn sure I would quickly and easily lose that case, despite the fact that google does not sell paper. And nobody would think that is weird, because Google is very obvious an invention (Yeah, I know, "googol", but the spelling makes it unique).
"Kik" is a lot closer to the "google" situation than the "apple" situation.
That's not necessarily true if Kik were planning to open up some kind of API. Also, the way the law works in the US encourages corporations to overreach. If you don't try to protect your trademark from every possible angle, it can lead to your loss of it in a court battle. A lot of times, lawyers will send out letters without caring how the recipient responds -- they only need to have sent the letter at all to have covered their bases.
Yep, I was going to say the same thing. As you say we can look up the US trademarks [1, 2] and see that this mark covers:
Computer software for use with mobile phones and portable
computing devices to:
- download audio, video, digital photos and programs;
- electronic payment systems, namely, a computer application
software used for processing electronic payments to and
from others;
- computer software for use with mobile phones and portable
computing devices to create video and digital photos to share
with other users; computer software for use with mobile phones
to launch other applications and connect to other software
services.
"Providing an interactive website featuring online non-downloadable software" indicates to me that there could indeed be confusion in a repository of software code to be used by other applications.
I'd suggest it's certainly not obvious over-enforcement.
They have the responsibility to defend their trademark, otherwise it could be abused by a rival claiming they did not adequately defend it. That said, NPM could have just said, "No, stop bothering us" and the lawyers might have backed down, satisfied their attempt to defend the trademark fulfilled their duty.
I applaud this action and while I'd like to point the finger at NPM, there's no real other method to fix historical package versions that depend on this.
It is worth pointing to the silly state of NPM packages: Who decided that an external dependency was necessary for a module that is 17 lines of code?
module.exports = leftpad;
function leftpad (str, len, ch) {
str = String(str);
var i = -1;
if (!ch && ch !== 0) ch = ' ';
len = len - str.length;
while (++i < len) {
str = ch + str;
}
return str;
}
Developers: less dependencies is better, especially when they're so simple!
You know what's also awesome? The caret semver specifier[1]. You could install a new, broken version of a dependency doing that-- especially when other packages using peerDependencies rely on specific versions and you've used a caret semver specifier.
Reading some of the comments reminds me old tale about a young man, that every morning on his way to work passed by a beggar and gave him a coin (that was back when coins actually had some value). One morning though the beggar notices the coin is smaller than usual, and he asks:
- Why you gave me a different coin today?
and the young man says:
- I got married and now I'm starting a family, I need more money so I can not give you as much anymore.
And the beggar cries out:
- People, look at this putz, he got married, and now I have to feed his family?!
I think the fact that we get so many awesome things for free is unbelievably lucky. I mean, not only we work in the one of the more generously paid jobs, we also get a lot of the tools we need for free! How cool is that? But some people think that if they are given those awesome things for free, they must deserve it and whoever gives them owes them forever. That's not the case. Yes, it is annoying to find somebody who contributed before does not want to do it anymore. It is mildly inconvenient and it can be improved. But let's not lose the perspective - the author does not owe us or npm continued support. It is sad he does not want to do it anymore, but that's what open source is about - people can take it over, and it happened within a single day. Such resilience is something to be proud of, not something to complain about.
FYI I'm the one who republished left-pad after it was unpublished.
I think of it similar to letting a domain name expire. The original author removed the code and I forked it and published a new version with the same package name.
The main issue was there were so many hard coded dependencies to 0.0.3 so I asked npm support if they could allow me to re-publish that version and they complied since I was now the maintainer of that package.
This is a surprisingly effective protest action. It got the attention of an incredible number of people very quickly, and the damage is mostly limited to wasting the time of a bunch of build cops.
I don't have much of an opinion on his actual reasons for protesting, but I do think it was a pretty cool protest.
Azer has contributed awesome modules to the community, but such a move _obviously_ messes with a bunch of people who previously didn't trust npm, but Azer. Npm works fine. There might be issues with it, but the reason builds are failing right now is that he decided to unpublish all of them - in a move that feels very kneejerky, despite him claiming that it's the opposite.
If this had been actually in the interest of the community (because he thinks that npm isn't acting in our interest), he'd give people a fair warning. I could have lived with a "Hey, this was my experience, it sucked, I'll unpublish things in 30 days. Please update your dependencies." We know how to deprecate things gracefully.
I am obviously a old fossilized ancient developer. This situation seems like insanity.
not the unpublishing part. the part where the thing that you require to sell/publish/do your job isn't under control or isn't stored within your organization.
Am i wrong in thinking that you should just have a local copy of all of your source code dependencies. would it really take that much longer?
This is why you should vendor it. What is "it"? All of it, whatever it may be. You should be able to build your systems without an internet connection to the outside world.
I say this with no reference to particulars of your language or runtime or environment or anything else. This is merely a specific example of something that could happen to a lot of people, in a lot of languages. It's just a basic rule of professional software development.
I agree. I use and love NPM and others like it, but when it comes down to it, i check-in my dependencies when my applications get "released".
Committing the updates is only one more step, and in my experience it's not even another step since we already have a rule that new installs or dep updates need their own commit.
I don't think "vendoring" in the sense of copying all your dependencies into your source tree is a good idea. Source and dependencies are different things and well worth using different tools for. You absolutely do want to ensure that your builds are reproducible even if the upstream libraries go away, but there are ways to do that without putting their source in your source tree.
I don't undertand why this isn't the most common reaction, maybe people don't want to take this much responsibility over their builds since they've grown used to it being "taken care of" by these central package repositories for them?
Still relying on internet connection for builds to pass seems rather silly to me.
Or have a decent package management setup? Or just use an existing one, it's been working for 20 years. Package management is a solved problem, just use an existing one and move on.
I've never felt good any time I have to use node modules and see this gigantic stream of dependencies come flying down. It's even more painful when you need to assemble license information for your software and crawl through _every single dependency and all of their dependencies_ to find their licenses, etc. to check they are OK to use in your software. Just look at the View License info in the Atom text editor some time for a truly insane wall of text (over 12,000 lines!!). IMHO the entire node / NPM system is seriously flawed with so many tiny dependencies for trivial stuff.
I think that unfortunately this was a foregone conclusion. Copyright law, like most other laws in our society, favor corporate interests.
I support his stand on principal, however. Azer is a talented developer and has an impressive life story, and has certainly contributed more to society than a social network well know for invading children's privacy.
You start a project with the same name as a company, which owns the registered brand and are surprised when some 3rd party complies with legal suggestions to make an adjustment?
Seems kind of silly to expect that NPM would want to fight for your project name when you didn't seem to do your own due diligence when picking a name. Also, a bit backwards to go remove all your modules as well, therefore breaking builds.
I've certainly benefitted from the vast ecosystem of npm. I greatly appreciate the work that goes into making this ecosystem what it is. However, I think we need to be a bit more critical when it comes to acquiring dependencies. Especially authors of very prominent packages.
Fun fact: one of my projects (a web api) depends on over 700 unique name/version modules.
If NPM wants to stay relevant and a serious contender, they need to have more clear policies in case of IP issues. In this case, the companies weren't even in the same space. Republishing someone's package who has chosen to unpublish and leave your platform is akin to Facebook resurrecting a Facebook profile because they had a lot of friends and the social circle ripple effects would be too high for feed quality for other users, so they chose to reactive the account AGAINST the author's wishes. WHAT?!?
We need an open source NPM alternative, yesterday.
> We need an open source NPM alternative, yesterday.
NPM is open source.[1] You don't want an open source NPM, you want a distributed consensus NPM where nobody has absolute power to censor or destroy other peoples work.
It seems like a prime use case for a DHT, possibly with some blockchain functionality identifying ownership of namespaces that people can reserve, like namecoin.
As best I can tell, before NPM restored it, Azer had transferred ownership to somebody else, who wanted to republish the old version but couldn't. So they do appear to be respecting the maintainer's wishes.
Wow, very interesting post for me. Earlier today, at work, we ran into an issue where `npm install` was failing because the `shuffle-array` module wasn’t found. Investigation showed that the cause was that it was unpublished today. We found that this was a required dependency of the `match` module and this was in our dependency list in`package.json`.
We investigated and found out that it had been erroneously committed — it’s actually a memory game and has absolutely no place in our webservice project. :) (Mistakes happen… dependency audits can be worthwhile!)
Now, some hours later, I found your post on HackerNews and was really shocked to see, hey, this is exactly why it was unpublished. Quite a chain of events. Never thought I’d figure out why the modules were unpublished, but now I get it! Thanks for the explanation.
funny thing, but assuming that kik is related to kik.com
if you look here http://dev.kik.com/build/, they promote their own server eg. "Our open source web server Zerver can help serve your cache manifest properly, as well as doing other speed boosting stuff like automatic style inlining."
I think it's amusing to see this from the perspective of the company. Some guy uses your trademark without your permission so you tell him to knock it off. He refuses, so you go around him, and so he protests... by fucking over all of his users. In a dispute that doesn't involve them. And people are celebrating this.
[+] [-] callmevlad|10 years ago|reply
Since these libs are now baked into various package.json configuration files (some with 10s of thousands of installs per month, "left-pad" with 2.5M/month), meaning a malicious actor could publish a new patch version bump (for every major and minor version combination) of these libs and ship whatever they want to future npm builds. Because most package.json configs use the "^1.0.1" caret convention (and npm --save defaults to this mode), the vast majority of future installs could grab the malicious version.
@seldo Is there a plan to address this? If I'm understanding this right, it seems pretty scary :|
[1] https://medium.com/@azerbike/i-ve-just-liberated-my-modules-...
[+] [-] _nfin|10 years ago|reply
[+] [-] overgard|10 years ago|reply
[+] [-] vjeux|10 years ago|reply
- Add 2 factor authentication for npm publish
- When you npm install, add a warning for all the versions that got published without 2 fac
- pre-install/post-install scripts should require user to accept or refuse. The simple action of running npm install shouldn't run arbitrary code.
- make shrinkwrap by default (and fix all the issues with it) so that running npm install doesn't use different versions when used over time.
- make updating a version an explicit decision via npm upgrade
[+] [-] scoot|10 years ago|reply
Too late. Every package name on the list has been claimed already by a randomer with unknnown intentions.
[+] [-] giancarlostoro|10 years ago|reply
[+] [-] jonahx|10 years ago|reply
[+] [-] spinlock|10 years ago|reply
IMO, this could end npmjs of they don't fix the issue.
[+] [-] erikb|10 years ago|reply
[+] [-] antihero|10 years ago|reply
[+] [-] namelos|10 years ago|reply
then please use named scope as default like@cycle/core @reactivex/rxjs.
[+] [-] ludamad|10 years ago|reply
[+] [-] krainboltgreene|10 years ago|reply
[+] [-] Too|10 years ago|reply
I would say the whole npm is pwned until these packages are either restored or that the package name is blacklisted/reserved. Is it possible to blacklist packages locally so I get warned if they are pulled in as sub dependencies? I don't trust every package maintainer to be quick enough to remove these dependencies.
[+] [-] z3t4|10 years ago|reply
I also think dependencies should be non-flat/tree/static. So that you only have to trust one dependency, and not all of it's child dependencies and their dependencies too. You should only need to manage your own dependencies, not other peoples dependencies.
There should also be a way to contact everyone who use your module, so you can tell them about critical bugs etc.
[+] [-] dre-hh|10 years ago|reply
[+] [-] joepie91_|10 years ago|reply
[+] [-] cliveb|10 years ago|reply
[+] [-] nordsieck|10 years ago|reply
For those who don't know, the purpose of trademarks is to prevent customer confusion; essentially we don't want people to be able to sell cheap knock-offs of someone else's thing without the general public being able to easily distinguish between them. In practical terms, trademarks are "scoped" by their "goods and services" declarations.
For example, Apple the device manufacture[1] and Apple the record label[2] could both be trademarked because they had non-overlapping goods and services declarations... until iTunes started selling music[3].
If you look at kik's trademark application[4], you can clearly see that the trademark is limited to chat/media consumer applications, a pretty obvious over enforcement.
[1] http://apple.com
[2] http://applerecords.com
[3] https://en.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer
[4] https://trademarks.justia.com/858/93/kik-85893307.html
[+] [-] vincent_s|10 years ago|reply
"KiK is the largest textile discounter chain in Germany and operates about 3,200 stores in Germany, Austria (since 1998), Slovenia and Czech Republic (since 2007), Hungary and Slovakia (since 2008), Croatia (since 2011) and Poland (since March 2012)."
https://en.wikipedia.org/wiki/KiK
[+] [-] overgard|10 years ago|reply
Smith is a very common surname. If one person starts the Smith Automobile Company and another person starts the Smith Farm, obviously there's no issue there. That's because none of us invented "Smith", it's understood to be a common name, etc.
On the other hand, if I start "The Google Paper Company", I'm pretty damn sure I would quickly and easily lose that case, despite the fact that google does not sell paper. And nobody would think that is weird, because Google is very obvious an invention (Yeah, I know, "googol", but the spelling makes it unique).
"Kik" is a lot closer to the "google" situation than the "apple" situation.
[+] [-] smt88|10 years ago|reply
[+] [-] jahewson|10 years ago|reply
[1] http://tmsearch.uspto.gov/bin/showfield?f=doc&state=4804:lir...
[2] http://tmsearch.uspto.gov/bin/showfield?f=doc&state=4804:lir...
[+] [-] delinka|10 years ago|reply
I'd suggest it's certainly not obvious over-enforcement.
[+] [-] tantalor|10 years ago|reply
[+] [-] larkinrichards|10 years ago|reply
It is worth pointing to the silly state of NPM packages: Who decided that an external dependency was necessary for a module that is 17 lines of code?
Developers: less dependencies is better, especially when they're so simple!You know what's also awesome? The caret semver specifier[1]. You could install a new, broken version of a dependency doing that-- especially when other packages using peerDependencies rely on specific versions and you've used a caret semver specifier.
[1] https://github.com/lydell/line-numbers/pull/3/files
[+] [-] smsm42|10 years ago|reply
- Why you gave me a different coin today?
and the young man says:
- I got married and now I'm starting a family, I need more money so I can not give you as much anymore.
And the beggar cries out:
- People, look at this putz, he got married, and now I have to feed his family?!
I think the fact that we get so many awesome things for free is unbelievably lucky. I mean, not only we work in the one of the more generously paid jobs, we also get a lot of the tools we need for free! How cool is that? But some people think that if they are given those awesome things for free, they must deserve it and whoever gives them owes them forever. That's not the case. Yes, it is annoying to find somebody who contributed before does not want to do it anymore. It is mildly inconvenient and it can be improved. But let's not lose the perspective - the author does not owe us or npm continued support. It is sad he does not want to do it anymore, but that's what open source is about - people can take it over, and it happened within a single day. Such resilience is something to be proud of, not something to complain about.
[+] [-] camwest|10 years ago|reply
I think of it similar to letting a domain name expire. The original author removed the code and I forked it and published a new version with the same package name.
The main issue was there were so many hard coded dependencies to 0.0.3 so I asked npm support if they could allow me to re-publish that version and they complied since I was now the maintainer of that package.
[+] [-] praxulus|10 years ago|reply
I don't have much of an opinion on his actual reasons for protesting, but I do think it was a pretty cool protest.
[+] [-] felixrieseberg|10 years ago|reply
If this had been actually in the interest of the community (because he thinks that npm isn't acting in our interest), he'd give people a fair warning. I could have lived with a "Hey, this was my experience, it sucked, I'll unpublish things in 30 days. Please update your dependencies." We know how to deprecate things gracefully.
[+] [-] jimjimjim|10 years ago|reply
not the unpublishing part. the part where the thing that you require to sell/publish/do your job isn't under control or isn't stored within your organization.
Am i wrong in thinking that you should just have a local copy of all of your source code dependencies. would it really take that much longer?
[+] [-] chvid|10 years ago|reply
[+] [-] cammsaul|10 years ago|reply
https://twitter.com/seldo/status/712414400808755200
[+] [-] jerf|10 years ago|reply
I say this with no reference to particulars of your language or runtime or environment or anything else. This is merely a specific example of something that could happen to a lot of people, in a lot of languages. It's just a basic rule of professional software development.
[+] [-] Klathmon|10 years ago|reply
Committing the updates is only one more step, and in my experience it's not even another step since we already have a rule that new installs or dep updates need their own commit.
[+] [-] lmm|10 years ago|reply
[+] [-] howeyc|10 years ago|reply
Still relying on internet connection for builds to pass seems rather silly to me.
[+] [-] antihero|10 years ago|reply
[+] [-] cyphar|10 years ago|reply
[+] [-] drinchev|10 years ago|reply
Is this a joke or something coordinated with the community?
[1] https://www.npmjs.com/~nj48
EDIT : The hijacked modules look suspicious. http://www.drinchev.com/blog/alert-npm-modules-hijacked/
[+] [-] tdicola|10 years ago|reply
[+] [-] jwiley|10 years ago|reply
I support his stand on principal, however. Azer is a talented developer and has an impressive life story, and has certainly contributed more to society than a social network well know for invading children's privacy.
https://medium.com/@azerbike/i-owe-my-career-to-an-iraqi-imm...
https://en.wikipedia.org/wiki/Kik_Messenger#Controversies
[+] [-] x0ner|10 years ago|reply
You start a project with the same name as a company, which owns the registered brand and are surprised when some 3rd party complies with legal suggestions to make an adjustment?
Seems kind of silly to expect that NPM would want to fight for your project name when you didn't seem to do your own due diligence when picking a name. Also, a bit backwards to go remove all your modules as well, therefore breaking builds.
[+] [-] joeandaverde|10 years ago|reply
https://www.npmjs.com/package/escape-string-regexp
I stopped searching at 1.
I've certainly benefitted from the vast ecosystem of npm. I greatly appreciate the work that goes into making this ecosystem what it is. However, I think we need to be a bit more critical when it comes to acquiring dependencies. Especially authors of very prominent packages.
Fun fact: one of my projects (a web api) depends on over 700 unique name/version modules.
Fellow programmers. This is embarrassing.
[+] [-] aioprisan|10 years ago|reply
[+] [-] zanny|10 years ago|reply
NPM is open source.[1] You don't want an open source NPM, you want a distributed consensus NPM where nobody has absolute power to censor or destroy other peoples work.
It seems like a prime use case for a DHT, possibly with some blockchain functionality identifying ownership of namespaces that people can reserve, like namecoin.
[1]: https://github.com/npm/npm
[+] [-] chc|10 years ago|reply
[+] [-] nchelluri|10 years ago|reply
We investigated and found out that it had been erroneously committed — it’s actually a memory game and has absolutely no place in our webservice project. :) (Mistakes happen… dependency audits can be worthwhile!)
Now, some hours later, I found your post on HackerNews and was really shocked to see, hey, this is exactly why it was unpublished. Quite a chain of events. Never thought I’d figure out why the modules were unpublished, but now I get it! Thanks for the explanation.
[crossposted from the medium article]
[+] [-] zwetan|10 years ago|reply
if you look here http://dev.kik.com/build/, they promote their own server eg. "Our open source web server Zerver can help serve your cache manifest properly, as well as doing other speed boosting stuff like automatic style inlining."
this Zerver is on github and build with npm
https://github.com/jairajs89/zerver/blob/master/package.json
I did not run the build but I'm pretty sure that now their server is not building anymore as it depends on babel
call that irony ;) ?
[+] [-] overgard|10 years ago|reply