Node seems to evoke strong responses from people. As someone who has a couple of side projects running on Node in production, and also as someone who considers himself fairly language/platform agnostic - I will say this:
For a lot of people - Node allows them to do a lot of things that was previously non-trivial in Ruby or Python land. From comet based applications to using socket.io for real-time notifications. It's not that these things can't be done in other platforms. There have been plenty of Java COMET solutions. Heck, 3 years back I wrote my own multiplayer strategy game in Erlang/OTP. However today, if I were to write that game - I will do that in Node. Reactor pattern is nothing new - but if you work with Node you will realize that for certain classes of problems, neither event machine nor twisted come close to Node and its ecosystem today.
Yes, C++ servers could be faster and JS can be a terrible language to work with. But a lot of us don't care - we want to get work done and we use Node for stuff that it's good at. There are always going to be people using it for everything - but every community has its set of zealots. Someone said the other day that he/she uses Haskell for prototyping.
For a lot of people - Node allows them to do a lot of things that was previously non-trivial in Ruby or Python land. ...for certain classes of problems, neither event machine nor twisted come close to Node and its ecosystem today.
This is the kind of stuff that causes the backlash. The JVM doesn't exist? Erlang doesn't exist? Why did the Node people insist on building a new ecosystem from scratch? Wouldn't it have been easier to build on something that was already mostly working (and already supported multicore)?
I'm curious, why would you write that game in Node now, if you already know Erlang?
From my perspective, Erlang is a better fit than Node for all the problem domains that Node is good at. I understand using Node if you don't know Erlang, but do know JS, but if you're already skilled with Erlang...
For a lot of people - Node allows them to do a lot of things that was previously non-trivial in Ruby or Python land. From comet based applications to using socket.io for real-time notifications. It's not that these things can't be done in other platforms.
One of the problems I have with the hype around Node is that people seem to gloss over the fact that asynchronous IO is only really dramatically more useful for the particular case of wanting to have a lot of connections that are open and idle most of the time. Many people, instead of saying “Node is great for comet because of asynchronous IO” say “Node scales really well because of asynchronous IO”, when in fact those 2 statements are very different.
Why is it that folks who point out that Facebook, Twitter, and Square all had to move to a "fast" language never seem to be able to point to any company (in the current generation) that achieved massive success by starting in a fast language? Having to transition to another language once you achieve success is a pretty solvable problem in comparison to actually creating a product successful enough that platform matters.
For that matter, neither Square nor Twitter have exactly "migrated" (I don't know about Facebook). They optimized hotspots -- a long and storied tradition. Even C programmers drop down into assembly.
Facebook still writes features in php, they just compile to C++ to eke out performance (i dont work there). I'm not sure where he got the "fixed" quotation from, the word 'fix' is not mentioned at all in the fb post to which he linked.
This article is spot on. We'd all like to have magical properties like speed and scale instantly conferred on our code merely by using the right tool. The fact of the matter is that both of those properties require consistent correct decisions at the application level to be achieved.
Scale, for that matter has been totally conflated with speed by many in the Node.js. A single-core reactor is not inherently scale-out. You can, of course, build a scale out app on node.js, but there's nothing about using node (or any other tool) that magically makes your app scaleable.
I think question that could be asked is, how long can any one tool, used correctly, defer the need to scale?
For some services, Node.js may give you more time and capacity. For others, Ruby or Python might.
You might need a static website, and you can develop it and push out the 'compiled' HTML to your server. For all the scaley goodness Node.js or whatever may seem to offer, for that purpose it'd never compete with NginX running a static server with appropriate caching.
Up until that point becomes visible on the horizon, you're just wasting time (and money) on a problem you don't or, if you're unlucky, might never have.
The reason I don't want to write much in node.js is simply I don't like callbacks. It's an inversion of control, and not a necessary one. One can easily invert control back to direct style by using futures/promises. There are even node.js extensions that do various conversions back to direct style, but since the language doesn't endorse any particular one you can't use them across a whole app (unless you write everything yourself).
The stack of callbacks piling up is definitely not a great feature of node.js code. You can mitigate it by using declared functions instead of anonymous functions but that also becomes burdensome when all your callback function does is call something else that also requires a callback.
This will be an interesting problem for node developers to solve. I like the Go approach of using channels. Twisted uses deferreds and inline yields depending on your preference. There are some libraries out there that people have written for node.js that also help.
I run a production node.js app and my experience hasn't been bad. My cofounder and I spent more time in the beginning (not much more time, mind you) learning how to deal with multiple outstanding async calls, a good application structure for express, etc. Now it's just as fast for us to go in and add a new API call that does this one query in node as it would be in another system.
The only technical problem I've seen that node is an easier solution is dealing with third party apis concurrently. It wouldn't be very straightforward to have a redis query, a mongo query, a facebook query, and a sendgrid query all open at the same time and managing the results when they all come in (and keeping program flow maintainable) in PHP, Python, Ruby, etc. Probably not a common problem or desired feature, though.
We're 100% Node on heroku with a custom buildpack, using a hosted Mongo service, some Node written cron jobs (which were pretty interesting to write), and redis and we serve 5 digits worth of hits every day at least, and I'm more than happy with performance. I never once look back and regret the decision to use node. I don't necessarily care about sharing code, having frontend developers do everything, etc like some node guys throw around naively, but I am looking forward to bringing somebody on that can grasp adding in a new API route, throw in a Mongoose find, and render a template with some variables. We have EJS templates serverside and clientside for that reason, and that could just as easily be done in PHP or Python.
However, I am enjoying working with node, the node community, and the interesting modules and stuff that are being released. I will say that there's a good bit more active "lets hack this up into a module and open source it over the weekend" kind of people in node than others at the moment.
callback is called when all of the others are complete. If you have dependencies, like facebook needs the output of redis and mongo then use async.auto which will automatically run things in parallel and in the order you need them.
I think part of the "problem" is that Node represents two different "radical" ideas together: first, using JavaScript on the server, and second, using evented/runloop programming. As such, the discussions become kind of confused. For example, not being someone too excited about Node myself, I still found this particular article kind of strange: I see no issue at all with using JavaScript on the server and in fact think that's the best part about Node. I am on the end of the spectrum where I don't even understand the point of discussing what language "deserves" to be on the server.
However, due to the hyper success of Node, we were left in this strange place where Node became the defacto JavaScript platform, and you can't really use JavaScript on the server/desktop without it (you can I guess, but don't expect anyone else to be able to use your code easily). As such, I think part of the reason you have people who want to use Node "for everything", is that a lot of them just actually want to use JavaScript "for everything", which is a less contentious issue in my opinion. In a world where you had "JavaScript on the server", and the optional "Node library" where you could do evented programming, vs. a different Rail/synchronous style library, you'd have something that looks a lot more like the other programming worlds.
I for example want to use JavaScript as my ideal scripting environment on the desktop, which is not that well suited for the asynchronous model of Node (just reading a bunch of files, operating on them, and then spitting out a new file can be kind of tedious with this model -- I'm not really waiting on anything or trying to hold a million open connections simultaneously). JavaScript itself I think competes just fine against Python and Ruby for these tasks, but again we can't really compare them in this abstract way, we have to compare JavaScript in the particular way it exists in Node, which brings along a lot of opinionated asynchronous APIs.
V8 isn't really designed to run multi-threaded code, which makes sense, since it's designed to run JavaScript code for the browser, and browser JS is single-threaded (except for Web Workers, which are isolated, and isolation requires a different style of code just like asynchronicity does). Here's one effort to add multithreading to V8: http://code.google.com/p/v8-juice/wiki/ThreadingInJS However, it has limitations because V8 isn't really intended to be used that way.
It seems odd to me that we had Javascript as pretty much the default server-side scripting language back in the Netscape days, then JS on the server pretty much disappeared as IIS and Apache took over, and now with Node Javascript on the server is "new".
I was going to use Node.js for a site I was building, but then I stepped back and realised going with Node.js (and MongoDB) meant I had absolutely no idea what I was doing.
I stepped back and used Python with Flask and SQLite as I had done before. There was nothing wrong with what I was familiar with. (An added bonus was I could actually reuse code from an older project using a similar architecture)
I initially had the worry that I had no idea what I was doing with Node.js and MongoDB. Honestly I still don't know what I'm doing. I wanted to learn these technologies eventually so I figured why not fumble through the process of learning and failing now so I'll be better with later project?
Keep in mind that my project isn't time or language sensitive. It's a practice project that could go somewhere with more work, but I can take all the time I need. I highly recommend a project like this if you want to learn these or any other new technology, otherwise you can keep with what you're familiar with.
I also consider language religion to be a red flag. The folks who wrote SICP said it better than I " Above all, I hope we don't become missionaries. Don't feel as if you're Bible salesmen. The world has too many of those already. What you know about computing other people will learn"
I've dabbled in node a bit and have an odd idea that Javascript is much like Python with C syntax and a bit of quirkiness. I've got a ~400 line program that parses a Debian packages file and lays out build dependencies - sadly I hadn't learned of germinate at the time.
And having to change languages to scale the website further because you've Made It can be a good problem to have...
This is a tangential point in the article but he mentions interviewing people and asking questions about arrays vs linked lists. I was wondering whether a lot of you are working on stuff that involves deep understanding of data structures, sorting algorithms, etc? I seem to see it in a lot of interview questions. Is that just a good topic to separate those with formal education from self-taught people, or are a lot of you really spending a lot of time working on those types of problems?
(To add to that, I should say that I did study data structures in college and know the answers to these questions, but I just personally never have to deal with that level of the code in my work.)
Knowing (in Python, for example) when to use a list, a tuple, a dict, a set or a deque makes a rather big difference, and is a skill I probably use almost every day at work.
See also this tweet from last night for a real-world application:
At my current job I do spend a lot of time caring about using the right data structures and algorithms to get decent performance. At my previous job, where performance was not as important, there were still places where this knowledge was required. For instance, we had something analogous to Google's instant search, where each keystroke in the search box would cause the results to change instantly. Implementing this incorrectly would have caused it to be very slow for the user.
I don't ask these questions to separate people with formal education from people without. People with formal education get these questions wrong quite often.
The article misses the biggest advantage: writing browser & server code in the same language is a huge win. Libraries can be shared. It is easy to move functionality back and forth. You can have one implementation of input validation logic instead of two.
I would love to see a concrete, documented example of this code sharing between the server-side and client-side. I'm not interested in "context switching" but I am extremely interested in how to actually share code between the two environments; I believe this is part of the key to creating more accessible content.
Could you point me to something I could look at further?
The author is not arguing whether there can be a server-side framework using a dynamic language like Javascript. Obviously there can, just like there is RoR, Django and others. His point is that the Node community came to such a point where they think Node is a silver bullet that can solve scalability issues just by itself.
It doesn't strike me as ridiculous to make a server-side Javascript framework. I'm not interested in Node because from code examples I've seen, its programming model seems to make doing even simple things quite complex. I really like Javascript, though.
[+] [-] karterk|14 years ago|reply
For a lot of people - Node allows them to do a lot of things that was previously non-trivial in Ruby or Python land. From comet based applications to using socket.io for real-time notifications. It's not that these things can't be done in other platforms. There have been plenty of Java COMET solutions. Heck, 3 years back I wrote my own multiplayer strategy game in Erlang/OTP. However today, if I were to write that game - I will do that in Node. Reactor pattern is nothing new - but if you work with Node you will realize that for certain classes of problems, neither event machine nor twisted come close to Node and its ecosystem today.
Yes, C++ servers could be faster and JS can be a terrible language to work with. But a lot of us don't care - we want to get work done and we use Node for stuff that it's good at. There are always going to be people using it for everything - but every community has its set of zealots. Someone said the other day that he/she uses Haskell for prototyping.
[+] [-] wmf|14 years ago|reply
This is the kind of stuff that causes the backlash. The JVM doesn't exist? Erlang doesn't exist? Why did the Node people insist on building a new ecosystem from scratch? Wouldn't it have been easier to build on something that was already mostly working (and already supported multicore)?
[+] [-] yosh|14 years ago|reply
From my perspective, Erlang is a better fit than Node for all the problem domains that Node is good at. I understand using Node if you don't know Erlang, but do know JS, but if you're already skilled with Erlang...
[+] [-] pilgrim689|14 years ago|reply
[1] http://www.ssrg.nicta.com.au/publications/papers/Klein_EHACD...
[+] [-] jberryman|14 years ago|reply
So...?
[+] [-] notJim|14 years ago|reply
One of the problems I have with the hype around Node is that people seem to gloss over the fact that asynchronous IO is only really dramatically more useful for the particular case of wanting to have a lot of connections that are open and idle most of the time. Many people, instead of saying “Node is great for comet because of asynchronous IO” say “Node scales really well because of asynchronous IO”, when in fact those 2 statements are very different.
[+] [-] kemiller|14 years ago|reply
For that matter, neither Square nor Twitter have exactly "migrated" (I don't know about Facebook). They optimized hotspots -- a long and storied tradition. Even C programmers drop down into assembly.
Use different pieces for what they're best at.
[+] [-] baudehlo|14 years ago|reply
[+] [-] perlgeek|14 years ago|reply
[+] [-] Androsynth|14 years ago|reply
[+] [-] munificent|14 years ago|reply
Didn't Google start in C++?
[+] [-] ckluis|14 years ago|reply
[+] [-] charliesome|14 years ago|reply
C programmers usually only drop down to assembly to do something that can't be done in C
[+] [-] andrewvc|14 years ago|reply
Scale, for that matter has been totally conflated with speed by many in the Node.js. A single-core reactor is not inherently scale-out. You can, of course, build a scale out app on node.js, but there's nothing about using node (or any other tool) that magically makes your app scaleable.
[+] [-] bsimpson|14 years ago|reply
> Google: "Run your app on our servers, and you'll scale really well, just like Google."
> Developers: "Sweet! Where do I sign up?"
> Developers: "Wait - my app is actually really slow."
> Google: "Duh! You have to write it exactly like we do to make it scale."
[+] [-] FuzzyDunlop|14 years ago|reply
For some services, Node.js may give you more time and capacity. For others, Ruby or Python might.
You might need a static website, and you can develop it and push out the 'compiled' HTML to your server. For all the scaley goodness Node.js or whatever may seem to offer, for that purpose it'd never compete with NginX running a static server with appropriate caching.
Up until that point becomes visible on the horizon, you're just wasting time (and money) on a problem you don't or, if you're unlucky, might never have.
[+] [-] noelwelsh|14 years ago|reply
[+] [-] btipling|14 years ago|reply
This will be an interesting problem for node developers to solve. I like the Go approach of using channels. Twisted uses deferreds and inline yields depending on your preference. There are some libraries out there that people have written for node.js that also help.
[+] [-] mcs|14 years ago|reply
The only technical problem I've seen that node is an easier solution is dealing with third party apis concurrently. It wouldn't be very straightforward to have a redis query, a mongo query, a facebook query, and a sendgrid query all open at the same time and managing the results when they all come in (and keeping program flow maintainable) in PHP, Python, Ruby, etc. Probably not a common problem or desired feature, though.
We're 100% Node on heroku with a custom buildpack, using a hosted Mongo service, some Node written cron jobs (which were pretty interesting to write), and redis and we serve 5 digits worth of hits every day at least, and I'm more than happy with performance. I never once look back and regret the decision to use node. I don't necessarily care about sharing code, having frontend developers do everything, etc like some node guys throw around naively, but I am looking forward to bringing somebody on that can grasp adding in a new API route, throw in a Mongoose find, and render a template with some variables. We have EJS templates serverside and clientside for that reason, and that could just as easily be done in PHP or Python.
However, I am enjoying working with node, the node community, and the interesting modules and stuff that are being released. I will say that there's a good bit more active "lets hack this up into a module and open source it over the weekend" kind of people in node than others at the moment.
[+] [-] dc-tech-fan|14 years ago|reply
Use a library like Async, it makes this really easy.
https://github.com/caolan/async
(assume redis, mongo, facebook, and callback are all functions)
async.parallel([ redis, mongo, facebook], callback);
callback is called when all of the others are complete. If you have dependencies, like facebook needs the output of redis and mongo then use async.auto which will automatically run things in parallel and in the order you need them.
[+] [-] tolmasky|14 years ago|reply
However, due to the hyper success of Node, we were left in this strange place where Node became the defacto JavaScript platform, and you can't really use JavaScript on the server/desktop without it (you can I guess, but don't expect anyone else to be able to use your code easily). As such, I think part of the reason you have people who want to use Node "for everything", is that a lot of them just actually want to use JavaScript "for everything", which is a less contentious issue in my opinion. In a world where you had "JavaScript on the server", and the optional "Node library" where you could do evented programming, vs. a different Rail/synchronous style library, you'd have something that looks a lot more like the other programming worlds.
I for example want to use JavaScript as my ideal scripting environment on the desktop, which is not that well suited for the asynchronous model of Node (just reading a bunch of files, operating on them, and then spitting out a new file can be kind of tedious with this model -- I'm not really waiting on anything or trying to hold a million open connections simultaneously). JavaScript itself I think competes just fine against Python and Ruby for these tasks, but again we can't really compare them in this abstract way, we have to compare JavaScript in the particular way it exists in Node, which brings along a lot of opinionated asynchronous APIs.
[+] [-] eurleif|14 years ago|reply
If you just want to write code in blocking style, you can use asyncblock with node.js: https://github.com/scriby/asyncblock
[+] [-] wtracy|14 years ago|reply
[+] [-] xtian|14 years ago|reply
[+] [-] TazeTSchnitzel|14 years ago|reply
I stepped back and used Python with Flask and SQLite as I had done before. There was nothing wrong with what I was familiar with. (An added bonus was I could actually reuse code from an older project using a similar architecture)
[+] [-] ndcrandall|14 years ago|reply
Keep in mind that my project isn't time or language sensitive. It's a practice project that could go somewhere with more work, but I can take all the time I need. I highly recommend a project like this if you want to learn these or any other new technology, otherwise you can keep with what you're familiar with.
[+] [-] shanemhansen|14 years ago|reply
[+] [-] happycube|14 years ago|reply
I've dabbled in node a bit and have an odd idea that Javascript is much like Python with C syntax and a bit of quirkiness. I've got a ~400 line program that parses a Debian packages file and lays out build dependencies - sadly I hadn't learned of germinate at the time.
And having to change languages to scale the website further because you've Made It can be a good problem to have...
[+] [-] jakejake|14 years ago|reply
(To add to that, I should say that I did study data structures in college and know the answers to these questions, but I just personally never have to deal with that level of the code in my work.)
[+] [-] dochtman|14 years ago|reply
See also this tweet from last night for a real-world application:
https://twitter.com/#!/djco/status/202490834472542210
[+] [-] anonymoushn|14 years ago|reply
I don't ask these questions to separate people with formal education from people without. People with formal education get these questions wrong quite often.
[+] [-] tlb|14 years ago|reply
[+] [-] bphogan|14 years ago|reply
Could you point me to something I could look at further?
[+] [-] mcs|14 years ago|reply
[+] [-] ecolak|14 years ago|reply
[+] [-] rmATinnovafy|14 years ago|reply
Anytime I stumble with a Node or Js or Rails guy the same question pops to mind. Why limit myself to one language or framework?
[+] [-] Locke1689|14 years ago|reply
[+] [-] edtechdev|14 years ago|reply
[+] [-] ilaksh|14 years ago|reply
[+] [-] derekorgan|14 years ago|reply
[+] [-] IsaacSchlueter|14 years ago|reply
[+] [-] mcginleyr1|14 years ago|reply
[+] [-] moron|14 years ago|reply
[+] [-] hackermom|14 years ago|reply