There is a fairly important point that is missed by this article. You do not need build tools to write JavaScript and depending on which tools you use there are significant drawbacks to using any them.
Before you dive into the world of build tools and process management I urge you to write unminified, ES5 (aka the JavaScript that runs without transpilation today) until it hurts.
Write unminified JavaScript and watch your page load times.
Write ES5 and time how long it takes you to complete projects of a given size.
Write reactjs code but use the JSXTransformer. Watch page performance. Watch how many times you reload the page in a given sitting.
It's really only when you find yourself with a problem that you can quantify that these tools start to make sense.
Discover for yourself why these tools exist or you'll waste a ton of time learning the newest thing and in the end not have gained much at all.
Think about the possibilities and limitations. (I personally love his approach at the beginning of projects when I could care less about fiddling with gulp and want to get into exploring the guts of a problem)
There's a definite balance between cargo-culting and reinventing the wheel. Minification is often a premature optimization. A somewhat proper/standardized module system (aka that which comes with ES6), on the other hand, is almost a necessity when it comes to building software.
Standalone JS was meant to add little bits of interactivity to documents. If what you're building is interactive documents, build tools are generally overkill. If you're building applications, the tools exist largely b/c JS-in-the-browser was not designed to build applications.
> Discover for yourself why these tools exist or you'll waste a ton of time learning the newest thing and in the end not have gained much at all.
Exatcly. It's important to clarify these things because so many inexperienced developers get lost in the maze of js tooling for no apparent reason.
Learning how to use these tools takes time which could be spent on learning how to program properly. And in some cases JS tooling isn't necessary at all, even in more complex projects. For example if you end up working in environments such as rails where most of these things are done automatically.
I've also noticed how people seem to look down on GUI tools such as https://incident57.com/codekit/. I think it's a great solution for beginners and people who don't need complex custom tooling.
"The biggest weakness of such small tools approach is that it is hard to learn what to use and how to configure it."
No. By far the biggest weakness of such a small tools approach is a Balkanization of development tools that generally refuse to work with one another and often don't work very well by themselves. One library I really wanted to try was using Browserify but I wanted to use brunch/bower because my workflow was already in brunch. Even with the brunch-browserify plugin this wasn't possible. After wasting two days on it, I gave up and just used Browserify by itself. Another couple of hours it was actually working. This is typical. On another app, I use gulp. Another piece of garbage that claims that when you run a command 'gulp watch' it will automatically rebuild your assets. It won't.
These are only three small projects, each using a different build tool: browserify, gulp, and brunch + bower. None of them are compatible with each other and it's unlikely they will ever work together in one project without hours or days of trial and error. If there was one monolithic (or not) dependency / build tool that actually worked, I'd much rather use that, and it would be a much better approach than having a whole bunch of small crappy tools that don't work together.
tl;dr: Not only is having multiple tools doing the same thing in this area not appealing, it just leads to developers wasting massive amounts of time and "what the fucks?" working with half a dozen tools that do the same thing and do it poorly.
That may be your experience but it's not even close to mine. I've used gulp, bower, and browserify across several projects and it's never taken me more a couple of hours to set them up and they have saved me a ton of time. You might not like the way the tools work but that certainly doesn't mean they are poor tools.
This was definitely my experience when I tried to move from python/django to javascript and gulp/browserify/foundation/nunjucks/backbone And it almost led to me getting fired. I almost certainly made rookie mistakes in trying to set up my development environment, but I think the lesson learned here is that when I try that again, I'm going to either:
1) Work with each tool individually first and really get to know it.
2) Have someone else who is experienced in this assemble the toolchain with the promise that I can come to them to debug toolchain problems.
My biggest hesitation about WebPack is that it does too much and duplicates so many other tools. If I'm already using Gulp to watch files and BrowserSync (love!) as my local dev server, why do I need WebPack when I'm only going to be using one of its abilities? Isn't this bloat?
We talk about "modern JavaScript tooling" but year after year, the list essentially stays the same.
Maybe few new players have appeared (Gulp, WebPack, Babeljs) but they do exactly the same thing that the tools we had before (e.g. Grunt, Browserify, Traceur).
It occurs to me that "modern JavaScript tooling" is growing only vertically (better tools to build, better tools to modularize, better tools to transpilation), but not horizontally. If we see some "brave new tool" on the scene, this tool will do exactly what previous tools did, only better.
I would like to have some decent tools for:
* analysing of project structure (e.g. dependencies between modules, graphs, trees etc.)
* code visualising (not toy! Gource is beautiful, but pretty useless. JSCity... I also don't see much use of it. I would see something that would allow me to draw some useful information from code visualisation. Something that would allow to understand better. But I see only beautiful animations and abstract 3D scenes)
* maintaining code (something that would allow me to conduct massive scale refactoring, or automatically convert code from one framework to another etc.)
* better editors for HTML/CSS, maybe even some decent WYSIWYG
Okay. Plain build systems and transpilers also are super useful. I think that Gulp, Babel.js, Browserify etc. are greeeat. But I think we need more. Something different. There is still room for innovation. Projects grow bigger and I think that we need something that helps us
* to understand easily new codebase
* to navigate codebase, conduct semantic search etc.
* to maintaining, refactoring etc.
I feel that some important tools are missing, not created yet.
> * better editors for HTML/CSS, maybe even some decent WYSIWYG
Editors? How about an IDE.
In the java environment I can manage an application container, profiler, debugger, compiler, packager, test suite all from one application.
Also included: near-omniscient auto-complete, hot-code replace, incremental building, dependency fetching, visual version control, automatic refactoring, deployment and many other conveniences I'm taking for granted. Hell, I could even file tickets from my IDE if I wanted to.
I think we'll see the emergence of some of these next-gen tools once ES6+ has started to bed in. ES6 modules are much more receptive to static analysis than ES5 code.
Esprima [1] is a full-featured and quite extensible ES5 parser, which has been used as the basis for a lot of the sort of analysis tools you're thinking about here. You might want to take a look at it, and at the software that's been written around it. In particular, there are several static analysis tools you might find of interest; I don't have a handy list, but searching "esprima static analysis" should find you plenty of candidates for further investigation. Refactoring tools shouldn't be hard to write, too, given an AST. (But I'd tend to suspect that automatic conversion from one framework to another is probably a pipe dream, at least for any nontrivial code base. Frameworks tend to come with too many mutually incompatible assumptions for that to be possible without an outright rewrite. Hell, that's even true of Angular 1.x and Angular 2!)
I'd be curious to know what you mean by "better", in your fourth bullet. I can think of a few things, but I doubt they're the same things you've thought of.
(And, while I have extensive experience with WYSIWYG HTML/CSS editors, that experience lies far in my professional past, because I found they were always too inaccurate to be worth the effort. Besides, it's not like there's anything particularly difficult about just viewing your changes in an actual browser; with tools like LiveReload, you needn't even go to the modicum of effort required to hit F5.)
What kind of codebase visualization would you consider "useful"? I think that's really the hard question to answer there; implementing a visualization is probably pretty easy, compared to coming up with a visualization from which you can easily derive information that's hard or impossible to obtain any other way. (I surmise this to be a difficult question based on the fact that no such visualization exists, or at least if it does it's not well known. Otherwise, it'd probably be part of the standard toolkit by now.)
Specifically for "analyzing of project structure" there does exist [MaDGe (Module Dependency Graph)](https://github.com/pahen/madge). You can generate some pretty nifty dependency graphs for your javascript codebase. I use it everyday when working on my game as I have a step in my gulp watch task that checks for any circular dependencies I may have introduced every time I hit save. It was a lifesaver when I made the switch over to browserify for my large game-codebase and found out I had a nightmarish dependency graph with a huge amount of cycles in it.
I think it's good to note that you don't actually need a single one of these tools to make functioning software. You can pack your scripts with cat and download libraries with wget and use regular old JavaScript without transpilers. When you have a problem with this, you can use tools... But you don't have to top-load every project with a whole suite of complex tools just for the sake of it.
No one needs anything. You can write code in notepad. No one wants that, and no one wants to have a million globals on the browser or hit F5 every single save any more. While there is a learning curve to something like gulp, you really only need to set it up once and you can use it on every project as simply as typing "npm i".
Like any tool, the value of these tools becomes apparent only when the alternatives become inconvenient. If your workflows are simple enough that you don't need them, more power to you!
Also, while JS lib/framework fatigue is a very real thing, ever since I switched to React+Webpack, I haven't felt the urge to switch to the 'latest n greatest'.
They are a major milestone in the JS dev tools space, and a pretty safe bet for someone who just wants to pick something and not have to think about it afterwards.
Well, you do at minimum need something that understands CommonJS modules, if you want to pack isomorphic code to run in the browser, so you need Browserify or Webpack.
And you need a module downloader that resolves and installs dependencies, so you need NPM and/or Bower.
And, if you want to write unit tests (which you should!) then you need a Javascript test harness, because otherwise you're going to have a bad time instantiating your modules and injecting mocks into them. You could do this in just plain Node, sure, but you'll end up reinventing a lot of wheels if you do. So you need Mocha, or something very like it.
But build tools are sort of a luxury, sure. You can get by just fine with make, if you already know how to use it. And nobody needs transpilers, except people who just don't have enough problems in their lives already.
If you want to get started with this stuff but you're feeling lost or overwhelmed then the free SurviveJS - Webpack and React book is a nice place to start! It covers many of these tools. http://survivejs.com
Yes, I feel like both Gulp and Grunt miss something fundamental that Make still has. I don't want to depend on other people's plugins for basic tasks and I want to only rebuild the stuff that changed. I tried Gulp with some 'notice if changed' plugins but it didn't seem to work so I end up sticking to Make.
Some of my Make build systems have gotten a bit unwieldy so recently I have been looking at using Ninja and ninja-build-gen from npm. This way I can still write my configure and my build tasks in JS/CoffeeScript and get to use a modern less cluttered version of Make that will scale well with the project.
I tend to agree that Gulp is better than Grunt. I prefer the code over configuration approach Gulp takes. The other benefits is that Gulp is streams based and will save you a lot of disk I/O over Grunt.
I recently tried switching from Grunt to Gulp and was expecting to see improvements in build times, but my builds actually wound up taking about 2x as long (on an ssd). Maybe I was doing something wrong, I don't know, but it was pretty disheartening. Here is my Gruntfile in case anyone cares: http://hastebin.com/wezizasiye.js
Make can be good but can also be quite tedious/verbose to get started. I actually prefer using the npm scripts and using the binary files in `./node_modules/.bin/{ webpack, babel, etc }`
Takes no time at all and there is no interface that can break like in gulp or grunt
The very good thing about module bundlers versus transpilers and task runners is that changing a single file doesn't result in a complete rebuild of the project; since the bundler maintains a model of the dependencies between files, it only needs to recompile the files that matter.
The downside is that now you have a whole other ecosystem of tool wrappers that tends to lag behind the tools themselves. They also tend to be JS specific, so if you have any other assets, you'll still need a regular task runner.
Another tool that becomes useful as your npm dependency list grows is npm-shrinkwrap. It is too easy to get large projects into a state that the existing developers can build and test, but break in production builds and for new developers. Being able to strictly version dependencies and control minor package updates can save you from debugging bad builds and losing new contributors. It's not a silver bullet, but can save you some frustration when packages deviate from proper versioning practices.
This a good overview of some popular tools for JavaScript development.
The thing that gets me about modern JavaScript development is the amount of modules you download to build your code. Currently I'm going through a PluralSights course for Gulp.js. Just to help me build and test my code, over 240MB of modules were downloaded. I'm starting to understand the benefits of all those modules. The thing that gets me is that the process makes you download all those modules again for the next project.
In my opinion Ember JS is the best framework out there. It's instantly productivity and everything just works out of the box. No worry about tooling and such. If you are coming from Ruby on Rails or similar platforms, ember JS is your best bet.
Very useful. This will be required reading for all our students :)
I created a react/grunt/browserify/babelify (+bootstrap) starter repo to clone from github for them but think it's still confusing. This provides much needed background information in one bundled place (even though the stack is slightly different and only mentions grunt).
The React world embraced WebPack after a modularity-kerfuffle back in January[1]. Pete Hunt followed that with his webpack-howto repo [2] which kind of turned the tide.
I really like Gulp/Browserify/Watchify/BrowserSync, but I'm starting to feel left behind and need to give WebPack an honest try.
Reading the page then the comments, even as a senior developer who only touches JavaScript when necessary; I don't see much unification in the JavaScript world today compared to 12 to 15 years ago.
React seems to be a theme, if I want to develop a modern (is Web 2.0 still the term) single-page site (is that still a phrase?) using HTML5/CSS3/JavaScript...what environment exists that is cohesive and complete as either Eclipse for Java or Visual Studio for .NET?
Here's my answer, as an outsider, Java dude who only suffers brief interludes with JavaScript...I'd pick jQuery Mobile or DART.
The thing is...whenever these JavaScript tooling articles comes up I don't see those listed. I see names like React, Meteor, Gulp, Grunt, Webpack and maybe 3 dozen more.
[+] [-] iandanforth|10 years ago|reply
Before you dive into the world of build tools and process management I urge you to write unminified, ES5 (aka the JavaScript that runs without transpilation today) until it hurts.
Write unminified JavaScript and watch your page load times.
Write ES5 and time how long it takes you to complete projects of a given size.
Write reactjs code but use the JSXTransformer. Watch page performance. Watch how many times you reload the page in a given sitting.
It's really only when you find yourself with a problem that you can quantify that these tools start to make sense.
Discover for yourself why these tools exist or you'll waste a ton of time learning the newest thing and in the end not have gained much at all.
[+] [-] sirgawain33|10 years ago|reply
Three exercises along the lines of the parent that I found particularly valuable:
1. Compare VanillaJS TodoMVC to your framework of choice
http://todomvc.com/
https://github.com/tastejs/todomvc/tree/gh-pages/examples/va...
What does the framework buy you? Is the framework-powered code easier to read? Easier to understand for a newcomer to the code base?
2. Read every line of Effective Javascript (it's short and eminently practical) and write out every code example.
http://www.amazon.com/Effective-JavaScript-Specific-Software...
There are about a dozen small errors in the code in the book, see if you can find them.
3. Read substack's alternative Javascript build flow:
http://substack.net/task_automation_with_npm_run
Think about the possibilities and limitations. (I personally love his approach at the beginning of projects when I could care less about fiddling with gulp and want to get into exploring the guts of a problem)
[+] [-] jowiar|10 years ago|reply
Standalone JS was meant to add little bits of interactivity to documents. If what you're building is interactive documents, build tools are generally overkill. If you're building applications, the tools exist largely b/c JS-in-the-browser was not designed to build applications.
[+] [-] oldboyFX|10 years ago|reply
Exatcly. It's important to clarify these things because so many inexperienced developers get lost in the maze of js tooling for no apparent reason.
Learning how to use these tools takes time which could be spent on learning how to program properly. And in some cases JS tooling isn't necessary at all, even in more complex projects. For example if you end up working in environments such as rails where most of these things are done automatically.
I've also noticed how people seem to look down on GUI tools such as https://incident57.com/codekit/. I think it's a great solution for beginners and people who don't need complex custom tooling.
[+] [-] sillygeese|10 years ago|reply
Good point. Another reason to be careful is that there are too many half-assed solutions to problems that might not even be real.
[+] [-] JustSomeNobody|10 years ago|reply
[+] [-] joesmo|10 years ago|reply
No. By far the biggest weakness of such a small tools approach is a Balkanization of development tools that generally refuse to work with one another and often don't work very well by themselves. One library I really wanted to try was using Browserify but I wanted to use brunch/bower because my workflow was already in brunch. Even with the brunch-browserify plugin this wasn't possible. After wasting two days on it, I gave up and just used Browserify by itself. Another couple of hours it was actually working. This is typical. On another app, I use gulp. Another piece of garbage that claims that when you run a command 'gulp watch' it will automatically rebuild your assets. It won't.
These are only three small projects, each using a different build tool: browserify, gulp, and brunch + bower. None of them are compatible with each other and it's unlikely they will ever work together in one project without hours or days of trial and error. If there was one monolithic (or not) dependency / build tool that actually worked, I'd much rather use that, and it would be a much better approach than having a whole bunch of small crappy tools that don't work together.
tl;dr: Not only is having multiple tools doing the same thing in this area not appealing, it just leads to developers wasting massive amounts of time and "what the fucks?" working with half a dozen tools that do the same thing and do it poorly.
[+] [-] jaegerpicker|10 years ago|reply
[+] [-] ricogallo|10 years ago|reply
[+] [-] afarrell|10 years ago|reply
1) Work with each tool individually first and really get to know it.
2) Have someone else who is experienced in this assemble the toolchain with the promise that I can come to them to debug toolchain problems.
[+] [-] joemaller1|10 years ago|reply
My biggest hesitation about WebPack is that it does too much and duplicates so many other tools. If I'm already using Gulp to watch files and BrowserSync (love!) as my local dev server, why do I need WebPack when I'm only going to be using one of its abilities? Isn't this bloat?
[+] [-] hex13|10 years ago|reply
Maybe few new players have appeared (Gulp, WebPack, Babeljs) but they do exactly the same thing that the tools we had before (e.g. Grunt, Browserify, Traceur).
It occurs to me that "modern JavaScript tooling" is growing only vertically (better tools to build, better tools to modularize, better tools to transpilation), but not horizontally. If we see some "brave new tool" on the scene, this tool will do exactly what previous tools did, only better.
I would like to have some decent tools for:
* analysing of project structure (e.g. dependencies between modules, graphs, trees etc.)
* code visualising (not toy! Gource is beautiful, but pretty useless. JSCity... I also don't see much use of it. I would see something that would allow me to draw some useful information from code visualisation. Something that would allow to understand better. But I see only beautiful animations and abstract 3D scenes)
* maintaining code (something that would allow me to conduct massive scale refactoring, or automatically convert code from one framework to another etc.)
* better editors for HTML/CSS, maybe even some decent WYSIWYG
Okay. Plain build systems and transpilers also are super useful. I think that Gulp, Babel.js, Browserify etc. are greeeat. But I think we need more. Something different. There is still room for innovation. Projects grow bigger and I think that we need something that helps us
* to understand easily new codebase
* to navigate codebase, conduct semantic search etc.
* to maintaining, refactoring etc.
I feel that some important tools are missing, not created yet.
[+] [-] the8472|10 years ago|reply
Editors? How about an IDE.
In the java environment I can manage an application container, profiler, debugger, compiler, packager, test suite all from one application.
Also included: near-omniscient auto-complete, hot-code replace, incremental building, dependency fetching, visual version control, automatic refactoring, deployment and many other conveniences I'm taking for granted. Hell, I could even file tickets from my IDE if I wanted to.
[+] [-] Wintamute|10 years ago|reply
[+] [-] aaronem|10 years ago|reply
I'd be curious to know what you mean by "better", in your fourth bullet. I can think of a few things, but I doubt they're the same things you've thought of.
(And, while I have extensive experience with WYSIWYG HTML/CSS editors, that experience lies far in my professional past, because I found they were always too inaccurate to be worth the effort. Besides, it's not like there's anything particularly difficult about just viewing your changes in an actual browser; with tools like LiveReload, you needn't even go to the modicum of effort required to hit F5.)
What kind of codebase visualization would you consider "useful"? I think that's really the hard question to answer there; implementing a visualization is probably pretty easy, compared to coming up with a visualization from which you can easily derive information that's hard or impossible to obtain any other way. (I surmise this to be a difficult question based on the fact that no such visualization exists, or at least if it does it's not well known. Otherwise, it'd probably be part of the standard toolkit by now.)
[1] http://esprima.org
[+] [-] scribu|10 years ago|reply
Here's an interesting tool for that: http://www.graspjs.com/ (structural search/replace)
[+] [-] khalilravanna|10 years ago|reply
[+] [-] OmarIsmail|10 years ago|reply
Flow is also great.
[+] [-] mbrock|10 years ago|reply
[+] [-] cozuya|10 years ago|reply
[+] [-] findjashua|10 years ago|reply
Also, while JS lib/framework fatigue is a very real thing, ever since I switched to React+Webpack, I haven't felt the urge to switch to the 'latest n greatest'.
They are a major milestone in the JS dev tools space, and a pretty safe bet for someone who just wants to pick something and not have to think about it afterwards.
[+] [-] aaronem|10 years ago|reply
And you need a module downloader that resolves and installs dependencies, so you need NPM and/or Bower.
And, if you want to write unit tests (which you should!) then you need a Javascript test harness, because otherwise you're going to have a bad time instantiating your modules and injecting mocks into them. You could do this in just plain Node, sure, but you'll end up reinventing a lot of wheels if you do. So you need Mocha, or something very like it.
But build tools are sort of a luxury, sure. You can get by just fine with make, if you already know how to use it. And nobody needs transpilers, except people who just don't have enough problems in their lives already.
[+] [-] heidar|10 years ago|reply
[+] [-] tete|10 years ago|reply
[+] [-] kasbah|10 years ago|reply
Some of my Make build systems have gotten a bit unwieldy so recently I have been looking at using Ninja and ninja-build-gen from npm. This way I can still write my configure and my build tasks in JS/CoffeeScript and get to use a modern less cluttered version of Make that will scale well with the project.
[+] [-] hyperhopper|10 years ago|reply
[+] [-] nailer|10 years ago|reply
[+] [-] devNoise|10 years ago|reply
[+] [-] tokenizerrr|10 years ago|reply
[+] [-] AdrianRossouw|10 years ago|reply
Anything extra was very easily managed by some simple Makefiles.
[+] [-] dauoalagio|10 years ago|reply
Takes no time at all and there is no interface that can break like in gulp or grunt
[+] [-] applecore|10 years ago|reply
[+] [-] guntars|10 years ago|reply
[+] [-] k__|10 years ago|reply
[+] [-] dominotw|10 years ago|reply
[+] [-] deckar01|10 years ago|reply
[+] [-] devNoise|10 years ago|reply
The thing that gets me about modern JavaScript development is the amount of modules you download to build your code. Currently I'm going through a PluralSights course for Gulp.js. Just to help me build and test my code, over 240MB of modules were downloaded. I'm starting to understand the benefits of all those modules. The thing that gets me is that the process makes you download all those modules again for the next project.
[+] [-] reycharles|10 years ago|reply
The process is called compilation, and the tools are called compilers!
[+] [-] TeMPOraL|10 years ago|reply
[+] [-] eibrahim|10 years ago|reply
[+] [-] kriro|10 years ago|reply
I created a react/grunt/browserify/babelify (+bootstrap) starter repo to clone from github for them but think it's still confusing. This provides much needed background information in one bundled place (even though the stack is slightly different and only mentions grunt).
[+] [-] aaronem|10 years ago|reply
[+] [-] talles|10 years ago|reply
I'm starting to see people talking more about webpack than the former...
[+] [-] joemaller1|10 years ago|reply
I really like Gulp/Browserify/Watchify/BrowserSync, but I'm starting to feel left behind and need to give WebPack an honest try.
[1]: http://blog.namangoel.com/browserify-vs-webpack-js-drama
[2]: https://github.com/petehunt/webpack-howto
[+] [-] jebblue|10 years ago|reply
React seems to be a theme, if I want to develop a modern (is Web 2.0 still the term) single-page site (is that still a phrase?) using HTML5/CSS3/JavaScript...what environment exists that is cohesive and complete as either Eclipse for Java or Visual Studio for .NET?
Here's my answer, as an outsider, Java dude who only suffers brief interludes with JavaScript...I'd pick jQuery Mobile or DART.
The thing is...whenever these JavaScript tooling articles comes up I don't see those listed. I see names like React, Meteor, Gulp, Grunt, Webpack and maybe 3 dozen more.
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] erokar|10 years ago|reply
[+] [-] if_by_whisky|10 years ago|reply
[+] [-] unknown|10 years ago|reply
[deleted]
[+] [-] fiatjaf|10 years ago|reply