First of all:
Thank you, yarn, for helping the community see the naked emperor. Deterministic builds by default are such an obvious (in retrospect) core requirement.
Couple questions:
Question 1:
Does anyone else who's been around more than a couple years share my view that Yarn : NPM :: IO.JS : Node?
IOW: healthy competition, catalyst for necessary change, ultimately a bridge or stopgap.
Question 2:
Any good comprehensive writeups on best practices?Committing lockfiles is a no-brainer. But what about globals? Per-node-version seems logical, but there are also semantic problems with "global" packages vs concept of executable binaries. I'd love to see a strong writeup outlining and defending a standard approach.
When io.js merged back into node.js the following happened:
1. Node.js was abandoned and replaced with io.js.
2. Io.js was relabelled as node.js.
3. The node.js project was put under open governance via the creation of the Node Foundation.
This doesn't really compare to npm:
* The npm-cli's name is using npm Inc's trademark: giving it away would leave the company with no name or create unwanted ambiguity between npm Inc the company and npm-cli the (then) community-owned open source project unaffiliated with the company.
* Yarn does not share any source code directly with npm-cli, it's a complete reimplementation of a similar feature set. It's currently backed by a mirror of the npm registry but that's the extent of the overlap.
* The Node Foundation already exists and Yarn is already owned by the community. I could see Yarn joining the Node Foundation and the Node Foundation replacing its bundled dependency on the commercially owned npm-cli but even then npm Inc has nothing to contribute to the process.
Personally I would much rather see Yarn join the JavaScript Foundation because it speaks to the broader appeal of Yarn outside the traditional "Node community" and the politics involved in the latter and its close ties to npm Inc.
FWIW I consider the close ties between the Node project and npm Inc nothing more than a historical accident that has resulted in a conflict of interest that is overdue to be resolved by dropping npm-cli from the official releases.
Now that yarn is no longer distributed using npm (although it's still possible to install it that way) that seems more realistic than ever.
Answer 1: Yes but they're not exactly analogous and I don't see them merging. Yarn is a complete rewrite, and the compatibility "surface area" is much lower than Node itself so there's not much pressure for them to merge. Yarn has been working essentially flawlessly for me, so npm-cli is really going to have to surpass yarn in some way for me to switch back.
Answer 2: I don't use global packages often. For binaries I generally add a "scripts" entry to package.json for common tasks, which automatically adds "./node_modules/.bin" to the PATH. For one-off tasks I just run "./node_modules/.bin/executable"
It's more accurate to say that npm v5 : npm :: io.js : node.
Same team, performing a big refactoring and significant improvement and modernization of some bits that were outdated and difficult to improve without a controlled demolition and rebuild.
The analogy breaks down, of course, because the governance didn't change, the project's relationship to its corporate backer wasn't standing in the way of progress, etc. (Except, I guess, that it might have gone faster if we'd had more money to hire more devs? But some of this was just a slog through a very old codebase with a lot of scar tissue, and it's hard to speed that up by putting more hands on it.)
Everything in npm 5 was literally planned years in advance. When we have this many people depending on a thing, we have to be careful about how we make drastic changes. Yarn was a strong signal from the community that we were on the right track, but it only seems like a "catalyst" when seen from the outside. Correlation is not causation, even when it lands first.
"Deterministic builds by default are such an obvious (in retrospect) core requirement."
I agree... I like the theoretical concept that our dependencies, could be automatically managed/upgraded behind the scenes via a declarative versioning system, like semver. That, in my view, was the paradigm behind not having defaulted to dependency graph snapshot (lock/shrinkwrap) by default. In practice, people don't follow semver correctly and autoupdating patch+minor versions can cause you a lot of pain. It would be cool to make versioning more tied to actual API signature, so that changes to the signature could not actually be published as minor/patch changes... but would be forced to use a major version bump in publish. I don't know how this would work particularly in a language like javacscript w/out a typed interface... but I'm just theorizing...
Yeah, I think that's a good comparison, except the upsides of yarn seemed much more obvious to the community than was the case with IOJS. There seemed a lot less controversy than with IOJS.
What a time those days were, feels like a million years ago already.
> Thank you, yarn, for helping the community see the naked emperor. Deterministic builds by default are such an obvious (in retrospect) core requirement.
We dropped npm in favour of yarn a while back, but I imagine we'll switch back once the stable release is out. Our needs are perhaps simpler than most, but the key issues we were having with npm at the time were directly solved by yarn:
1. Dependency install was taking too long.
2. Inconsistent builds between devs because of no lock file.
3. Could not search the registry from the command line in a timely manner.
Having just done a quick test on a random project, I'm pleased to see that all of those concerns are now taken care of, plus the install (for this project at least) is ~30% faster than it is with yarn.
Just tried on a couple of projects with a lot of dependencies, we moved to yarn a while ago due to performance issues and it seems to be resolved.
On cold cache:
Yarn: 20.94 seconds
NPM5: 21.11 seconds
With cache:
Yarn: 10.35 seconds
NPM5: 15.20 seconds
For some reason, when node_modules folder is still there, yarn exits in a couple hundres milliseconds but npm5 does something for around 5 seconds.
Haven't checked lock file / installation consistency stuff. Yarn has been great on that too so we have no intention to go back but this is a decent release.
So performance is now comparable, which is awesome, but I'd still stick to yarn because we have been burned too many times by npm v2/v3 with call stack issues and other errors. I don't have the energy (or the time/faith) to go through that again.
Competition is wonderful though. I'll check npm again in 6-12 months and see what the community thinks of it before switching again.
So happy with the --save by default. Someone at work kept installing new dependencies without save (they didn't knew about it, somehow). We then had an unusable package.json. I had to manually find directories in node_modules and install them on production -_-.
You should implement basic ci with travis et al. That commit should never be able to make it into master. If you are using Github you can force green tests before a pull request is mergeable.
And I think this is a complete bs. I have to try the module and then make a conscious decision to use (save) it, not kind of save it first and then hope that it actually does what I want.
Like wtf, who thought that saving something you download maybe for the first time as a dependency is a good idea?
It bothers me somewhat that they say this is standardized, but not by whom. Is the idea that Yarn should fall in line or is it a case of https://xkcd.com/927/ ?
I want to be able to search by tag, constrained by license and with packages that use dependencies I already have installed floated to the top. There are so many ways this could be nicer.
Yarn doesn't support all of npm's features. At least not as of a couple months ago when it 1) didn't support one of our needs at all, and 2) broke a couple NPM packages on install. This may or may not be part of why it's faster: it does less.
This seems to be very similar to Gradle vs Buck (or Bazel).
Generally speaking the main tool is slow, or otherwise suboptimal but does a lot. A different tool is built with an emphasis on speed. Eventually, the main stream tools picks up the speed features of the other tool and then "wins" because it does more. (Side note: I have no idea when Gradle came out in comparison to Buck/Bazel) For context, Gradle in the 4.0 version is starting to really support build cache support.
Edit: this comment is a reaction to all the other comments I see here about yarn vs npm.
It installs two node.js projects (react & ghost) and shows how long it takes to do under multiple scenarios (cold cache, installed and lockfile). It is automatically run each day. It also creates an average if one version is run multiple times.
I never used yarn, but I just did some research about the changes in comparison to yarn.
Well, actually they are more similar now, obviously (saving by default for example). In terms of speed, npm has improved as well (https://twitter.com/passcod/status/860437637764571140). Even though npm still seems to be slower than yarn (https://docs.google.com/presentation/d/1re-AGrTOf4zmnXXDMTG1...). At the end, I'd say it's about personal preference, they both get the job done.
>since npm@3, npm will automatically update npm-shrinkwrap.json when you save
All installs update shrinkwrap then? doesn't that start to make it redundant to package.json in the first place now. Does this make git tracking package and shrinkwrap mandatory in-case you want to install, test but then roll back the version, as shrinkwrap will already have changed.
edit: Oh I'm supposed to --no-save now for that described flow?
This is probably a naive question, but the only use case I've come across of needing a flat-install option was to use Polymer.js web components (which is why so much of the polymer project relies on bower).
Aside from that, what is the use case that flat installs solve?
> A new npm cache verify command that will garbage collect your cache, reducing disk usage for things you don’t need (-handwave-), and will do full integrity verification on both the index and the content
> Running npm while offline will no longer insist on retrying network requests. npm will now immediately fall back to cache if possible, or fail. (#15666)
"package-lock.json" can someone please define what it is and does, I've heard of shrink wrap, google doesn't reveal anything about this file, I have a vauge idea that it has something to do with shrinkwrap but don't know why it exists or if it's supposed to replace shrinkwrap or whatever. splainit!
[+] [-] chrisweekly|8 years ago|reply
Couple questions:
Question 1: Does anyone else who's been around more than a couple years share my view that Yarn : NPM :: IO.JS : Node?
IOW: healthy competition, catalyst for necessary change, ultimately a bridge or stopgap.
Question 2: Any good comprehensive writeups on best practices?Committing lockfiles is a no-brainer. But what about globals? Per-node-version seems logical, but there are also semantic problems with "global" packages vs concept of executable binaries. I'd love to see a strong writeup outlining and defending a standard approach.
[+] [-] pluma|8 years ago|reply
1. Node.js was abandoned and replaced with io.js.
2. Io.js was relabelled as node.js.
3. The node.js project was put under open governance via the creation of the Node Foundation.
This doesn't really compare to npm:
* The npm-cli's name is using npm Inc's trademark: giving it away would leave the company with no name or create unwanted ambiguity between npm Inc the company and npm-cli the (then) community-owned open source project unaffiliated with the company.
* Yarn does not share any source code directly with npm-cli, it's a complete reimplementation of a similar feature set. It's currently backed by a mirror of the npm registry but that's the extent of the overlap.
* The Node Foundation already exists and Yarn is already owned by the community. I could see Yarn joining the Node Foundation and the Node Foundation replacing its bundled dependency on the commercially owned npm-cli but even then npm Inc has nothing to contribute to the process.
Personally I would much rather see Yarn join the JavaScript Foundation because it speaks to the broader appeal of Yarn outside the traditional "Node community" and the politics involved in the latter and its close ties to npm Inc.
FWIW I consider the close ties between the Node project and npm Inc nothing more than a historical accident that has resulted in a conflict of interest that is overdue to be resolved by dropping npm-cli from the official releases.
Now that yarn is no longer distributed using npm (although it's still possible to install it that way) that seems more realistic than ever.
[+] [-] tlrobinson|8 years ago|reply
Answer 2: I don't use global packages often. For binaries I generally add a "scripts" entry to package.json for common tasks, which automatically adds "./node_modules/.bin" to the PATH. For one-off tasks I just run "./node_modules/.bin/executable"
[+] [-] IsaacSchlueter|8 years ago|reply
Same team, performing a big refactoring and significant improvement and modernization of some bits that were outdated and difficult to improve without a controlled demolition and rebuild.
The analogy breaks down, of course, because the governance didn't change, the project's relationship to its corporate backer wasn't standing in the way of progress, etc. (Except, I guess, that it might have gone faster if we'd had more money to hire more devs? But some of this was just a slog through a very old codebase with a lot of scar tissue, and it's hard to speed that up by putting more hands on it.)
Everything in npm 5 was literally planned years in advance. When we have this many people depending on a thing, we have to be careful about how we make drastic changes. Yarn was a strong signal from the community that we were on the right track, but it only seems like a "catalyst" when seen from the outside. Correlation is not causation, even when it lands first.
[+] [-] cphoover|8 years ago|reply
I agree... I like the theoretical concept that our dependencies, could be automatically managed/upgraded behind the scenes via a declarative versioning system, like semver. That, in my view, was the paradigm behind not having defaulted to dependency graph snapshot (lock/shrinkwrap) by default. In practice, people don't follow semver correctly and autoupdating patch+minor versions can cause you a lot of pain. It would be cool to make versioning more tied to actual API signature, so that changes to the signature could not actually be published as minor/patch changes... but would be forced to use a major version bump in publish. I don't know how this would work particularly in a language like javacscript w/out a typed interface... but I'm just theorizing...
[+] [-] rattray|8 years ago|reply
However, npm could certainly compete with more of yarn's features (fast local cache[0], shorter syntax for running scripts like `npm start`, etc)
[0] turns out they did that: https://twitter.com/maybekatz/status/865393382260056064 wow!
[+] [-] draw_down|8 years ago|reply
What a time those days were, feels like a million years ago already.
[+] [-] spankalee|8 years ago|reply
[+] [-] nailer|8 years ago|reply
https://docs.npmjs.com/cli/shrinkwrap provides deterministic builds and has been around far longer than yarn. Since Oct 2014 npm v3 started automatically updating shrinkwrap whenever '--save' was used. See https://github.com/npm/npm/pull/4918#issuecomment-61344871
Calling npm 'the naked emperor' for not implementing something it did, infact, implement is uncalled for.
Edit: added links in response to unexplained downmods. The parent is simply, provably wrong.
[+] [-] 2manyredirects|8 years ago|reply
[+] [-] sschueller|8 years ago|reply
[+] [-] eknkc|8 years ago|reply
On cold cache: Yarn: 20.94 seconds NPM5: 21.11 seconds
With cache: Yarn: 10.35 seconds NPM5: 15.20 seconds
For some reason, when node_modules folder is still there, yarn exits in a couple hundres milliseconds but npm5 does something for around 5 seconds.
Haven't checked lock file / installation consistency stuff. Yarn has been great on that too so we have no intention to go back but this is a decent release.
[+] [-] gtirloni|8 years ago|reply
Competition is wonderful though. I'll check npm again in 6-12 months and see what the community thinks of it before switching again.
[+] [-] ryanar|8 years ago|reply
[+] [-] abritinthebay|8 years ago|reply
[+] [-] KayL|8 years ago|reply
npm@5 28.274s vs 11.60s Yarn
@Windows10, SSD, warn cache
[+] [-] Achshar|8 years ago|reply
[+] [-] Thomaschaaf|8 years ago|reply
[+] [-] gear54rus|8 years ago|reply
Like wtf, who thought that saving something you download maybe for the first time as a dependency is a good idea?
[+] [-] felixrieseberg|8 years ago|reply
Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
npm i -g npm-windows-upgrade
npm-windows-upgrade --npm-version latest
[+] [-] evolve2k|8 years ago|reply
Surely it would be much better to follow standard lock file naming conventions and name the file package.lock
[+] [-] qntmfred|8 years ago|reply
[+] [-] kuschku|8 years ago|reply
[+] [-] mstade|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] blitmap|8 years ago|reply
I want to be able to search by tag, constrained by license and with packages that use dependencies I already have installed floated to the top. There are so many ways this could be nicer.
[+] [-] synthecypher|8 years ago|reply
[+] [-] ashark|8 years ago|reply
[+] [-] shadowmint|8 years ago|reply
Edit: nevermind, I found it --> https://github.com/npm/npm/releases this is the 'prerelease' release.
[+] [-] swsieber|8 years ago|reply
Generally speaking the main tool is slow, or otherwise suboptimal but does a lot. A different tool is built with an emphasis on speed. Eventually, the main stream tools picks up the speed features of the other tool and then "wins" because it does more. (Side note: I have no idea when Gradle came out in comparison to Buck/Bazel) For context, Gradle in the 4.0 version is starting to really support build cache support.
Edit: this comment is a reaction to all the other comments I see here about yarn vs npm.
[+] [-] andyfleming|8 years ago|reply
[+] [-] Thomaschaaf|8 years ago|reply
The different shows how much better npm@5 is :)
It installs two node.js projects (react & ghost) and shows how long it takes to do under multiple scenarios (cold cache, installed and lockfile). It is automatically run each day. It also creates an average if one version is run multiple times.
[+] [-] pajoda|8 years ago|reply
[+] [-] Already__Taken|8 years ago|reply
>since npm@3, npm will automatically update npm-shrinkwrap.json when you save
All installs update shrinkwrap then? doesn't that start to make it redundant to package.json in the first place now. Does this make git tracking package and shrinkwrap mandatory in-case you want to install, test but then roll back the version, as shrinkwrap will already have changed.
edit: Oh I'm supposed to --no-save now for that described flow?
[+] [-] romanovcode|8 years ago|reply
[+] [-] scrollaway|8 years ago|reply
[+] [-] k__|8 years ago|reply
Is it finally save to install 2 times and get 100% the same packages?
[+] [-] nailer|8 years ago|reply
[+] [-] plexicle|8 years ago|reply
[+] [-] JeuelyFish|8 years ago|reply
Aside from that, what is the use case that flat installs solve?
[+] [-] bluetech|8 years ago|reply
[+] [-] floatboth|8 years ago|reply
Nice.
[+] [-] assafmo|8 years ago|reply
Finally! :-)
[+] [-] tomxor|8 years ago|reply
[+] [-] mstijak|8 years ago|reply
[+] [-] Achshar|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] kotojo|8 years ago|reply