top | item 37985779

Yarn 4.0

170 points| winterqt | 2 years ago |yarnpkg.com

139 comments

order
[+] d357r0y3r|2 years ago|reply
Anecdotally, it seems like a lot of Yarn adoption is still suck on 1.22.x. Migrating to 2+ appears to be untenable for many teams.

I understand that Yarn 2 is essentially a new tool, as there are certain features you simply can't ship while still being compatible with npm, but this hasn't resonated with developers. The initial appeal of Yarn 1 was that you could pretty much drop it into your node project and get much faster install times. `pnpm`, from what I understand, is faster than yarn, while still having that seamless interop w/ npm.

[+] Timon3|2 years ago|reply
While I personally enjoy Yarn 2+ a lot and prefer it for private projects (I'll explain a bit about my setup at the end), for a long time both the docs and the implementation pretty much guaranteed that it would fail:

- many basic things were missing from the docs for a long time, or at least were easier to find in Github issues (e.g. "what should I gitignore")

- defaulting to PnP meant that, without configuration, things wouldn't properly work on many projects based on frameworks/toolkits/editors

- opinionated defaults made things harder than necessary (e.g. why does `yarn init` create a Git repository?)

You either had to accept/work around these issues, or configure everything to work as e.g. npm or yarn 1 did. I did the latter, and am still happy that I bought in - but I can very much understand anyone who didn't.

My preferred setup with Yarn 2+ projects is:

- use the node-modules linker (I tried pnp every couple of months and always ran into some issue, which I sometimes sent PRs for, but there was always something not working)

- gitignore as described for zero-installs[0] (even though I don't use pnp; this has the advantage that you'll have a ZIP copy of every module cached, leading to very fast local/CI installs and resistance against registry issues)

- create .gitattributes file as described at the end of [0] to hide the binary files from diffs

- add the base SDK[1] for WebStorm support (this rarely worked well)

- add the TypeScript plugin

[0]: https://yarnpkg.com/getting-started/qa#which-files-should-be... [1]: https://yarnpkg.com/getting-started/editor-sdks

[+] madeofpalk|2 years ago|reply
I cannot express how painful yarn pnp has been for us. We've been stuck on a relatively pretty old version of Typescript (4.8) because upgrading it with yarn is so difficult. You have to contend with Yarn patching Typescript, and requiring obscure IDE integrations so it all just works. We still can't update, because we can't get Typescript + Yarn PnP + VS Code on macOS to work together. It has been such a massive time sink for us.

I see no reason for anyone to use Yarn. Either use vanilla npm, or use pnpm which doesnt require a mountain of hacks to work.

[+] h1fra|2 years ago|reply
On top of the really different API, you always get v1 when you install (which I never understood). You need to upgrade manually everywhere with a command line. If I had to bet: most user didn't even bother upgrading because they are not aware of this behavior.
[+] ForkMeOnTinder|2 years ago|reply
Yeah I followed the same path. I originally switched from npm to Yarn 1 for the speedup, but then Yarn 2 broke enough things that I never managed to upgrade.

pnpm is (at least) as fast as Yarn 1, and they managed to do it with a much better compatibility story (in my experience)

[+] eyelidlessness|2 years ago|reply
I’m not stuck on Yarn 1, I’ve chosen to keep using it because it’s the best tool for my usage. I tried Yarn 2 and found it unsuitable, because:

- it patches packages to work with its PnP system

- even if you opt out of PnP

- in ways that are opaque for me as a user

- and in ways that break usage, eg it broke a beta version of TypeScript because the patch it wanted to apply wasn’t available for the beta

I’m sure the PnP feature is brilliant, but I don’t want to use it and I don’t want its behavior to be applied when I don’t use it. So Yarn 2+ is not an option for me. And since Yarn 1 is still supported, and meets my needs well, I continue to use it by choice.

[+] kylemh|2 years ago|reply
fwiw the change specifically in this release (removing a lot of plugins) will make that adoption path a lot smoother.

Weirdly, going from v1 to v4 is a lot easier now than v1 to v2 or v1 to v3.

That being said... It's too little too late. I use `pnpm` now and maybe I'll use `bun` after total Node compatability.

[+] HatchedLake721|2 years ago|reply
Am I the only one that after doing Node.js for ~10 years never had much issues with npm or a need for yarn?

Yes, if memory serves me right, yarn might've been first with `yarn.lock` before `package-lock.json` was a thing.

But why would I use yarn today?

I don't care about speed of npm install locally, I do it once in a blue moon.

I don't care if my pipeline takes 3 minutes (yarn) to install dependencies instead of 5 minutes (npm).

[+] lucideer|2 years ago|reply
> But why would I use yarn today?

I think many many many people switched to Yarn 1.x wholesale and then switched back to NPM once it had undergone some improvements.

[+] a13o|2 years ago|reply
I view yarn as collective leverage over npm's product roadmap.

As long as the npm roadmap doesn't stray too far from what developers want, or ship those things too slowly, there should be no need to use yarn.

But for that to work, yarn needs to remain a credible alternative, so that it's actually possible to overcome the switching cost.

[+] MuffinFlavored|2 years ago|reply
Recently ran into a project that uses https://pnpm.io/

Add that to the mix...

In a similar vein,

https://github.com/lerna/lerna

I am not sure when to use `npm workspaces` instead of lerna now? It seems like the answer (for me at least) was "you don't need lerna anymore given recent (past 2 years) NPM updates"

[+] joduplessis|2 years ago|reply
Workspaces is where npm falls apart.
[+] gryn|2 years ago|reply
the reason to try yarn for me was berry pnp to avoid those pesky node_module folders but alas the experience has been bumpy on personal projects. for something that has promised more robustness, it has been anything but that.

I still like the yarn cli more than the npm one, but I ended up just defaulting to npm.

[+] ruffrey|2 years ago|reply
Same here. Npm seems to have fixed the lock file issues it had, which was the main reason. With caching, npm is fine. On large projects npm ci can take 2 minutes - we leverage node modules cache on the lock file hash, and it’s typically fast.
[+] meiraleal|2 years ago|reply
Npm served well but recently I've being noticing that I don't need NPM at all. For the frontend ESM modules from urls works very well. For the backend, deno can be a good alternative if it means not needing to use npm anymore.
[+] specialp|2 years ago|reply
Yarn made the same mistake going from 1 to 2 as Angular did. The newer version was so different. Both should have been named something else. At a certain point of incompatibility and divergence it is a different product, and the benefit you get from using a well known name is outweighed but the cost of confusion.

Yarn 2 brought a lot of great improvements but since it was named "Yarn" people were expecting it to behave somewhat like the first one.

[+] diggan|2 years ago|reply
Sometimes I wish for a SemVer iteration that does away with the major version, and requires a project to create a new library wholesale if they want to increment the major version. Every library would have minor and patch releases, but if they want to substantially change the API, they'll have to create a new library instead and ask people to move to it instead.
[+] microflash|2 years ago|reply
Yarn lost the plot when going to v2. It was very different from v1. Even attempting a migration was painful enough to write it off completely in the projects I was working on. I've yet to come across any new projects using Yarn since then. Now, I think it is too late. pnpm has more or less caught up and even surpassed any advantages Yarn had, with a much more pragmatic compatibility across multiple platforms.

Personally, I've moved on to pnpm (with npm as a fallback when I can't use pnpm).

[+] k__|2 years ago|reply
Haven't used this until today, but I browsed the code and the docs and it all made a very solid impression.
[+] solatic|2 years ago|reply
We switched already to pnpm and won't look back.

> Hardened Mode, constraints engine

a) pin your dependencies, set save-prefix='' in .npmrc, use a tool like pnpm outdated, Renovate, Dependabot, npm-check-updates to keep them updated. Doesn't need to be done or enforced in the package manager, Git Hooks and CI are sufficient. b) use syncpack to ensure all your dependencies in different workspaces use the same versions. This doesn't need to be done in the package manager.

pnpm also has pnpm licenses to help keep compliance happy.

If someone wants to write a package manager in Rust for speed (pnpm is already moving in this direction, see https://github.com/pnpm/pn and https://github.com/pnpm/pacquet ), we'd take a look. Otherwise - not enough benefit to switching away.

[+] __jonas|2 years ago|reply
I'm just trying to switch to pnpm and I'm a bit lost on how to dockerize packages in my workspace, something that I thought would be a common thing to do.

The example in their docs [1] seems to just be wrong

> FROM common AS app1

> COPY --from=prod-deps /app/packages/app1/node_modules/ /app/packages/app1/node_modules

> COPY --from=build /app/packages/app1/dist /app/packages/app1/dist

This of course will not work, since the node_modules folders of packages in a pnpm workspace just contain symlinks to the virtual store at the root of the monorepo, not the actual modules. I don't see any good way to do what this example pretends to achieve (minimize docker image build size) since even when hoisting is forced, pnpm seems to only ever hoist node_modules in the root of the repo.

Sorry for this somewhat unrelated rant, I was just surprised at hitting such an obstacle immediately after trying to adopt pnpm after I heard so much praise for this tool.

[1] https://pnpm.io/docker#example-2-build-multiple-docker-image...

[+] erhaetherth|2 years ago|reply
> If someone wants to write a package manager in Rust for speed

How about Zig?

Seriously, try Bun. It's stupid fast. Even if you don't use all its other features, just try "bun install".

[+] cantSpellSober|2 years ago|reply
> Hardened Mode [...] doesn't need to be done in the package manager

Why not?

[+] gavinray|2 years ago|reply
After the Yarn 2 debacle, I've been using Yarn 1 (as well as most people who use Yarn that I know).

I briefly tried Yarn 3 but it also did some weird things.

Here's to hoping that Yarn 4 "just works" the way Yarn 1 does.

[+] moomoo11|2 years ago|reply
Same. I was using yarn 1 until a couple years ago. Now I just use npm. What are the downsides? I have no idea. If there are, they must be negligible because the apps work fine, customers are happy, and devs are able to keep building. We must not have any crazy dependencies because I don’t see much performance gains. At most we might link other packages as we develop.

I think if I really went searching for problems I might find something. I also just try to avoid JS these days unless it’s UI, and there I keep things very lean and straightforward.

[+] apineda|2 years ago|reply
Same situation. I use npm on most of my newer projects and yarn on one project where I tried to upgrade it to 2 but it was pure disaster.
[+] rtsao|2 years ago|reply
Yarn 2.0 was ambitious, but in hindsight, it probably would have been better to make regular node_modules the default rather than pushing PnP and zero installs (and alienating most users in the process).

I think Yarn Berry with node_modules linker is strictly better than Yarn 1.0, whereas PnP and zero installs involve tradeoffs that might not be right for everyone. There's a lot of great design decisions in Berry that gets muddled with all the PnP-related discussion. I will say Maël deserves a lot of credit for driving corepack, which makes version control of package management totally seamless. It's always been a total nightmare but now it's shockingly easy.

[+] solardev|2 years ago|reply
Yarn's versioning is really confusing. Is 4.0 compatible with 1.x? Is it a successor to Yarn 2 (with the different functionality)? What happened to 3? Haven't really kept up with this since 2 wouldn't work with any of my projects, not sure where to pick up again now...
[+] jxi|2 years ago|reply
As a Bun convert, it's interesting that they felt the need to have a preemptive response for Bun as part of this major release. The response itself seems unconvincing though: Basically, Bun is a lot faster and simpler, but they think they can catch up.

I'm not convinced they can catch up even on speed, and Bun ergonomics are also a lot nicer from the get go.

[+] deliriumchn|2 years ago|reply
> Basically, Bun is a lot faster and simpler

Funny enough, I got our work project working with yarn berry (whatever number this was this summer) in 10-15 minutes, but I couldn't manage to do the same with bun. I'll try again now after they got few minor updates probably just to see if its better now...

[+] solardev|2 years ago|reply
Were you using Bun on a new project? I tried it on a few existing ones and couldn't get it to build on any of them
[+] doodlesdev|2 years ago|reply
Great release which solves the biggest painpoints with the latest Yarn versions. I still like zero-installs and yarnPath providing Yarn for anyone using the project, but not having zero installs and using corepack to provide Yarn seems like the right choice which reduces friction for adoption.

Also, not having to install plugins such as interactive-tools is sooooo good. I have no ideia why these are plugins in the first place as they are so essential to using Yarn.

I still wish PnP faced better adoption though. In most projects I still go back to using the pnpm or the node-modules linker because of various issues with PnP which are sometimes hard to debug.

In other notes, why the hell in Node corepack still experimental and not enabled by default? I simply don't get it.

[+] nu11ptr|2 years ago|reply
As an outsider to the JS world, can someone give me the quick pros and cons between yarn and npm? Can you switch back and forth in the same project? Is the end result of your package structure the same?
[+] acemarke|2 years ago|reply
The end result is essentially the same, yes - all directly requested dependencies, and all of their transitive dependencies, are extracted and installed on disk into `./node_modules`.

Yarn came out at a time when NPM was particularly slow and buggy. NPM has caught up some since then, but I _personally_ still like Yarn better. I find the CLI output more useful, and it feels like it installs in a more consistent amount of time.

You should only have one of them in use in a project at time.

PNPM is another alternative package manager that installs the same packages, but tries to use symlinks to a single globally-cached copy of each package to save disk space. (I believe recent versions of Yarn have an equivalent option, but it's not on by default.)

[+] smithcoin|2 years ago|reply
They each generate their own lockfiles that contain the dependency graph so they can’t be used together. They have different utilities in the CLI for example- NPM has npm audit, which yarn doesn’t. They’re mostly the same but you have to keep in mind they have different philosophies in how to manage the dependency graph.
[+] solardev|2 years ago|reply
TLDR: Choose one, stick with it, try not to have multiple node versions per machine/VM if you can help it, and update & test your packages in small frequent increments if you can. Otherwise it gets to be a nightmare real quick.

Yarn 1.x came out because NPM was really slow and buggy back in the day. Yarn was fast and buggy. Then Yarn 2 came out, was even faster and less buggy, but not backward compatible. I've never actually seen it used in the wild, even now (it's just yarn and NPM on all the projects I've worked on).

Bigger differences will come trying to upgrade multiple packages at once, especially across major versions (e.g. 2.x to 3.x). It's going to be hard no matter what, but `yarn upgrade-interactive` makes it a bit simpler, while npm has third party plugins that do similar things. There's also complexities involved if you need to switch the underlying `node` version across projects, since npm and yarn both need different node versions depending on their own version.

Their final outputs are similar (a node_modules folder where all your JS dependencies and sub-dependencies live), but their internal workings are different, and you shouldn't use both together because they will conflict with each other (sometimes).

Generally, switching back and forth isn't hard on smaller projects: just delete the lock files and delete the node_modules folder and reinstall everything from scratch.

But you really don't want to get into the habit of doing that. There's no real benefit (just choose one and stick with it; either is fine these days), and you may introduce hard-to-catch bugs related to one of your thousands of dependencies. Especially on bigger projects, erasing the lockfile means that it will try to use the designated SemVer in your package.json to get the desired version, but that may be different than the actual version that was being used, which is stored in the lockfile as a manual override. I'm not sure why these package managers had two sources of truth, but it's a nightmare.

[+] dboreham|2 years ago|reply
0. JS world is full of people captivated by shiny things.

1. yarn was a shiny version of npm

2. They did a Python 3 stunt (well...much worse) with yarn 2 so nobody uses it and everyone stuck with 1.x

3. Bugs aren't fixed in 1.x

4. So back to npm, or the next shiny thing.

Edit: no you can't switch back and forth. The tools end up storing internal state in your project repo. I mean, yes you can change, but it'll mean some hassle and possibly re-writing build files.

[+] IceDane|2 years ago|reply
Yarn 2 was just a colossal mistake. In nearly any real project it causes constant churn and is just a perfect example of what happens when you let some computer scientist do whatever they feel is nice from a theoretical perspective rather than based in reality.

There isn't even any arguing with it being an abject failure. It's just abundantly obvious when you look at how many people keep using yarn 1. These patch notes even prove it since they realized how insane it is to make their zero install thing the default.

[+] jauntywundrkind|2 years ago|reply
The new JavaScript powered constraint engine looks amazingly simple & elegant.

We definitely suffer badly trying to make the very simple example shown happen across our projects: how do we make sure everyone is using the same version of React (for example)?

I'd be curious to know how this rules engine is usable outside of Yarn. The post talks about it being used at Datadog, seemingly on their app. But I didn't see or missed info on the package itself & how it might be embedded elsewhere.

Also super notable from this release, turning off zero-install by default. Still such a neat idea. Even though most package managers are reasonably fast now, there's still often multiple hundreds of MB of files copied out of cache, for each project, and the idea of just directly using the cache without copying stuff in feels like a real nice to have, a potential big saver of disk space.

[+] stephen|2 years ago|reply
Kinda surprised by all the "stuck on Yarn v1" comments so far...

We went through the same "~2015 everyone uses yarn", "~2020 everyone goes back to npm" cycle, but in the last ~year or so are back on yarn v3 for...reasons that I forget (oh yeah, the ability to download multiple platform libs for when you're running in docker [1])...but :shrug: it's been great.

My only complaint is that `yarn dedupe` should be automatic (as it was in yarn v1), or at least via a flag. :-D

But otherwise yarn v3 (with the node_modules linker) has been great, and look forward to using v4.

[1] here's the "afaict npm can't do this?" snippet we use in `.yarnrc.yml`: https://gist.github.com/stephenh/43c7f15b97b73baf821f2a22c62...

[+] depr|2 years ago|reply
Skipped v2, v3 works fine for me as well.
[+] madjam002|2 years ago|reply
Wow so much negativity here, I just can’t imagine not having Yarn PnP for my project with 15+ packages all in a monorepo. It’s super easy to patch 3rd party packages, fast at installing and adding new dependencies, and it plays so nicely with Nix which is great for CI/CD, as with a bit of tooling all of your packages will be cached between builds.

Even some “heavyweight” frameworks which I didn’t expect to work, work fine, such as NextJS.

I’m a very happy Yarn Berry user.

[+] jimrandomh|2 years ago|reply
I'm also in the "still using Yarn 1.x" camp. The problem is that yarn is installed to the global path by a package manager from outside the Javascript ecosystem, which means that the same install of yarn is shared between all the projects I might run on my system, including projects I don't maintain myself and including projects that aren't maintained at all. I'm open to other JS package managers if they offer benefits, but the one that runs when I type "yarn" in my shell has to be backwards-compatible with 1.x, forever. Repositories like Debian/apt-get and homebrew operate on a similar philosophy, and still offer only 1.x.

So... why didn't they just put a package-manager version number in package.json? I would have no problem with projects requiring Yarn 2+, if upgrading to Yarn 2+ didn't create an obstacle to continuing to run everything else that still uses Yarn 1. As it is now, I'm not willing to even _try_ the newer versions, because I expect to have to revert and I'm worried that I'll lose a day to the mess that the uninstall/revert will create.

[+] k__|2 years ago|reply
lol,

I just installed Yarn for the first time and got version 4.0.0, what a coincidence!

A clean major version? At this time of the year? In this part of the world?

I've never been more suspicious of a tool :D

But the codebase is really nice, especially the comments in the plugin system.

I'm SO gonna extend that thing.

[+] bhouston|2 years ago|reply
I find there is less room for yarn in the ecosystem than back when npm was crazy slow.

In my testing, npm has gotten a lot faster that it used to be and often it is faster than yarn 3.x when doing installs in a CI system. npm also has decent workspace support, excluding the important ability to do topological sort-based builds (which I still use lerna for.)

I also found bun install also works great and fast and is a drop in replacement for npm.

[+] wilg|2 years ago|reply
I came here to complain about how there are two JS package managers that are virtually identical, and learned there is a third. Fellas, sort this out. This is a waste of human potential.

(I also still think package managers should be language and ecosystem independent. We just need one way of getting named and versioned folders of files from the internet.)

[+] smithcoin|2 years ago|reply
Getting yarn berry to work was a nightmare in my org. Hopefully this release can actually simplify the toolchain.
[+] acemarke|2 years ago|reply
I'm curious, what problems did you run into?

My own experience is that Yarn 2/3 have worked great, as long as you stick with the "`node_modules` linker" option rather than the "PnP/Zero Install" option.