Huge no. of files for Angular 2

[paradite]

Interesting fact that I recently came across:

bower and many other npm packages, has some dependencies that eventually depends on a package called "wordwrap".

And this "wordwrap" package somehow has its test folder exposed in npm.

The result:

Every single person using bower would have one or more copies of In Praise of Idleness by Bertrand Russell on your local machine, depending on how many of your projects has a npm dependency of this package:

https://github.com/substack/node-wordwrap/blob/master/test/i...

Don't believe me? Try searching for "In Praise of Idleness" in Spotlight.

Edit: Someone had already sent a PR about this on GitHub: https://github.com/substack/node-wordwrap/pull/14

[pjc50]

It's a good job they didn't pick the Anarchists' Cookbook.

[aaronharnly]

Very nice! The file can also be found by searching for "Saran finds some mischief", which fees like an apt phrase.

[zuck9]

I used to `touch node_modules/.metadata_never_index` to prevent Spotlight from wasting disk cycles by indexing that stupid folder. After searching "In Praise of Idleness", it doesn't seem to prevent it. :/

Anyone knows how to prevent all node_modules folders from getting indexed by Spotlight?

[heydenberk]

I have 16 copies on my work computer.

[sotojuan]

I wouldn't be hopeful about that PR. Substack is awesome but he's really bad at replying to notifications because of the tons of modules he has.

[percept]

Wasteful yet subversive

[manimalcruelty]

0 copies. too lazy to work.

[justinsaccount]

This person is counting the node_modules directory. While JS is a bit insane and this directory will have a ridiculous number of files, they are concerned:

"because my deployment (google app engine) allows only 10K files"

meaning, they don't realize that node_modules is for development and not related to the application they would actually deploy.

[sotojuan]

Hope this comment stays at the top before all the "wow JS sucks!!!" people arrive :-) Though to be fair a "modern" JS dev environment does use a ton of stuff!

IIRC Angular 2 production builds are actually pretty efficient.

[DHMO]

> and not related to the application they would actually deploy.

this.

Granted, it's the same with everything- if you expanded every compressed file and counted every class, most languages and frameworks would just be incredibly nuts, because all of them aren't needed.

But, something could be done about that; you could better differentiate what is there for convenience, and what needs to be there and make the developer more aware of what they are using. It can make development more difficult if done poorly, but good examples of minimalist development are out there, e.g. Sinatra and similar frameworks that said X is too much- just use this.

[agconti]

Evaluating a framework by the number of files its dependencies are broken into is a a pretty poor measure of quality.

[huuu]

Why? If framework X does roughly the same as framework Y which is 50% in size then you can estimate that the code of Y is more effective.

[royka118]

I did a bit of investigation into angular2 dependancies. It has alot more than angular1 and react

http://royka.github.io/2016/05/03/front-end-deps.html

[k__]

It seems to me, that angular2 has a second system problem.

[keeganjw]

I started working with Laravel not long ago and found my project folder had 24,000+ files in it. And those don't compile down before you deploy... it makes me feel like I'm working on the tip of an unstable iceberg. Who the hell knows what's going on down there. No one person could possibly hope to know what it all actually does.

[ishener]

32,000 files for a hello world... jeez, i'm going back to java.....

[j_jochem]

Not a lot better:

  $ unzip -l /usr/java/jdk1.8.0_77/jre/lib/rt.jar | wc -l
  20138

It's just less of a burden on the host filesystem because those files are usually loaded straight from the jar (i.e. zip file).

[stupidcar]

No, 32k source files for the whole Angular 2 SDK. The deployable hello world app would be one JS file.

Try exploding every Jar in the JDK and counting how many class files and resources there are.

[moderndeveloper]

babel 6 with jsx transformer used to install a comparable number of files due to module duplication. At one point it was a 100M install with some modules being duplicated 45 times. Much of this was the fault of npm 2. But with latest babel and npm 3 it's now a 35M install with 5700 files over 814 directories. I guess that's considered lean by modern standards.

[moderndeveloper]

I've switch from babel 6 to buble recently. buble runs three times faster and its install size is 4.6M with 212 files over 39 directories. The install of buble is literally just "npm install buble" and it runs out of the box without configuration. Competition is a good thing.

[maaaats]

Just because of the way npm2 ordered the dependencies, the runtime of babel got incredibly slow [1]. Npm3 fixed that drastically, but I wonder how much is still wasted just because of navigating the file tree to the dependencies.

[1]: https://github.com/babel/babelify/issues/206

[currywurst]

Is there a "distribution" bundle convention for npm ? Analogous to static linking, it would be one .js file that would bind all dependencies into a bundle (e.g. gulp.dist.js) . In that case you would end up with a much smaller number of dependency files to manage.

[Klathmon]

There is for your final output (meaning the stuff you would upload to the server and serve to the user). but not for the development.

IMO it's a pretty big anti-pattern to do that. It just hides the problem of managing dependencies (see, it's not 10,000 files, it's just one!), but doesn't fix any of the issues associated with it.

Keeping each dependency small, and having tons of them means that deduplication can work better, tree shaking works better, and it lets you do things like swapping out one package for another with the same API.

[jbverschoor]

Why isn't npm managing packages like ruby gems?

SHARED_DIR/npm_modules/NAME/VERSION

[Klathmon]

Because package-lookup and the package manager are 2 completely separate systems in javascript.

When you `require` or `import` a file in node.js, it looks for a node_modules and looks for that name in there. If it can't find it there, it starts walking up the directory tree until it finds something it can use (to a point).

This is hardcoded and will be extremely difficult to change without a crazy amount of breaking.

The package manager is free to install however, but it needs to put things where the package-lookup can find them.

[inaccessible]

Reminds me of this talk that I just watched: https://www.youtube.com/watch?v=k56wra39lwA

[fernandopj]

At first glance, I thought the title was "Huge NO: on files for Angular 2" - a vulnerability report on filesystem capabilities of Angular 2 and why it should be abandoned

[douche]

Has npm finally figured out how to de-dupe dependencies? In one project, I have something like 47 copies of the same version of the same library, distributed at all levels of the node_modules hierarchy.

I try not to think about that JS tooling too hard, lest I start pulling my hair out and devolve into a screaming crazy person.

[rplnt]

This sucks so hard. It's also quite easy to hit path length limits for certain operations (on windows) when you have this mess there.

[Touche]

It's actually not possible due to the way Node looks up dependencies. You could have a situation like:

  node_modules/
    a/
      node_modules/
        c/ (1.0.0)
    b/
      node_modules/
        c/ (1.0.0)
    c/ (2.0.0)

Both a and b depend on c version 1.0.0, but since there's a version 2.0.0 in the root node_modules folder c can't be placed there, and has to be duplicated in a and b's own node_modules folder, otherwise Node couldn't find it for each of them.

[DCoder]

npm v3 tries to dedupe packages during installation to reduce this problem. However, it still does not install packages in a deterministic way...

[0]: https://docs.npmjs.com/how-npm-works/npm3-dupe

[1]: https://docs.npmjs.com/how-npm-works/npm3-nondet

[k__]

https://github.com/alexanderGugel/ied

[ivan_gammel]

To me it looks like a silly architectural mistake made by NPM/Node developers, considering that there's already pretty good dependency management solution on the market that does the things right.

Maven repositories have following structure, that allows to avoid duplication and take versions into account: /<vendor namespace>/<library>/<version>/<library artifact.ext>

Vendor namespace itself is hierarchical and usually related to domain name, e.g. "com/mycompany/myapp".

No idea, why this approach is not yet used in JS world (except the webjars), but it's high time to fix it this way.

[guigar]

Yes, that's already solved with npm 3.

[andrewclunn]

Okay, but when you run the build process, how big is the resulting distributable?

[talmand]

My results of the hello world tutorial in Angular2 was 53 requests, 4933.49KB, and loaded in 1.74s on local dev, according to browser dev tools. All for one html file that had one h1 element.

Plus, it started out broken. I had to search elsewhere to find the solution to the error the tutorial produced.