I honestly find "callback hell" a lot easier to follow and understand than the vast majority of fixes everyone is coming up with.
They're just continuations, seriously, what's everyone's problem? You define a function, it gets access to the current scope, it defines the rest of the program flow.
If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
It's not difficult, it's just gross. And these promises don't solve the main problem, which is that synchronous functions return the result, until at some point you add some IO, so now it takes a callback. And everything that calls it now has to take a callback. And hours later all you've done is add a network call in some basic function but your diff looks like a total re-write.
If they were really continuations they'd be fine. But they're not. They're a broken, terrible approximation of continuations.
A real continuation captures the entire execution context including the current call stack.
Instead what you get in javascript is a stack that's completely meaningless. There's no way to automatically route your exceptions to the real calling context, and there's no automatic way for you to be sure you're seeing all of your callee's exceptions.
If you really want to be sure you'll hear about the ultimate success or failure of an asynchronous call, every single asynchronous step needs to manually install it's own exception handler, trap any exceptions, and pass them back via a failure callback. You're basically doing by hand what the runtime environment is actually supposed to do for you (that being the whole point of exceptions).
And you do this when you have 10 different steps and each one of those steps should also have an error case. And you do this for 100s of resources and you have callback hell.
If you start seeing do1() do2().. or cb1(), cb2() and so on function that is the "hell" everyone is talking about. Of course you should name your functions better -- but that's not the point. Logically you might not need an extra function but you are forced to add it because of the way your framework forced you to handle IO.
>They're just continuations, seriously, what's everyone's problem?
That they're badly made continuations, without support from the language.
>If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
At the cost of moving stuff out of where it's invoked, so making code harder to read.
The problem with callback hell is that is pushes the programmer to write FOR the machine, in the way the machine likes it. Those things should be an implementation detail, and in good languages, are.
I mostly agree, every time I see one of these I find trying to read it as a transformed version of the vanilla callback version is the easiest way to understand it. After all, when callbacks do get out of hand, approximating one of these is exactly how I end up dealing with it.
I wish `let` was a little different, syntactically. With something closer to what OCaml does, you don't need to surround everything in parens and braces. This would take all the visual grossness out of it (which I think is the biggest problem people really have):
let (cb = function(x) {
...
}) {
foo(x,y,z,cb);
}
vs.
let cb = function(x) {
...
} in
foo(x,y,z,cb);
It doesn't look like much, but when you have a number of nested callbacks to define the latter syntax keeps everything on the same level where the former nests just as poorly as defining them in the parameter list. (I also hate seeing '})' anywhere, but not many people seem to care about that so much. It's why I switch to Monaco if I am doing JS.)
This is what I have being thinking. These proposals for ways to refactor always back themselves up by using actual use case examples of callbacks vs toy examples of their fix
"there! See how much cleaner that is?"
No, I don't, and this stories solution in particular, is particularly ugly.
Good overview of jQuery Deferred and how to use promises (at least the jQuery flavor). Promises (or futures) are a simple concept: an object-level abstraction of a non-blocking call, but they're very powerful when you see them in action. For example, the $.when method:
Let's say you have 3 ajax calls going in parallel. With $.when, you can attach callbacks to arbitrary groupings of those ajax calls (callback1 runs when ajax1 and ajax2 are done, but callback2 runs when ajax1 and ajax3 are done).
I first learned about promises in Trevor Burnham's excellent book Async Javascript (http://pragprog.com/book/tbajs/async-javascript) and it is still the best explanation of promises I've ever read. If you like this article and are interested in reading further about promises or the asynchronous nature of Javascript in general (both for browser and node.js), I highly recommend you check out this book.
I dont find that promises really help callback hell that much, they are useful but in the case of doing a series of sequential async functions the result is pretty similiar (the promises version is usually longer)
I got to write some firefox only code recently and added a delay to a function by just adding
... some code
setTimeout(continueFun, 5000)
yield;
... more code
it felt like magic, I dont like generators and would much prefer to see message passing and blocking calls like erlang, but failing that it will be nice to be able to use generators more regularly
Promises are not an alternative to callbacks - they are a wrapper around them that make working with them much nicer (though still a little painful as you imply). They add composability which I think has wider implications than many realize at first.
Consider that with things like jQuery deferreds I can start a request in one function and pass it around adding handlers in others...something very hard to do with the old CB model. Maybe later on you have some optional action that you only want to occur if a previous action had completed. This sort of thing is easy with deferreds. It makes bundling together disparate actions easier as well as handling series of things as you mention.
Having an abstraction layer in there also has a lot of potential that is (somewhat) untapped. There is a lot of possibility for library authors to add excellent error handling or debugging capabilities in there - things that are currently a bit painful with POC (plain 'ol callbacks).
I agree generators have strong possibilities in this area- I would note thought that they are not an alternative to promises - the two are orthogonal, and in fact they work quite well together. Just take a look at what some other Mozilla folks are doing with http://taskjs.org/ for example.
I made a library called async.js (https://github.com/eligrey/async.js) that makes it even easier to abstract away all callbacks with yields, so all you'd have to do is yield to.sleep(5); continueFun(); for your example.
More complex things are also easy to do as well, such as implementing a node.next(event) method, so you can simply do var click = yield document.next("click"); instead of the whole event listener shebang. The feedback example (https://github.com/eligrey/async.js#asking-the-user-for-thei...) shows how much of a difference using yield can be versus callbacks or even promises.
By the way, it is absolutely possible to implement a message passing style on top of generators instead of some random ad hoc resumption policy. Check it out:
I have used it in browser games to load assets (images, sounds, data files) before starting a new state (menus, gameplay, etc). It's a fantastic library.
It's a real shame to have a language this high level, yet still have to go through this much crap just to get things done. Manual memory management is easier than this. But while including GC in the runtime has it's drawbacks, there is no reason that a language can't just handle task switching for you (like Go does, for example).
IMHO the event model / callback spaghetti / what-have-you is tricky precisely because it operates at a high level of abstraction.
Memory management, by comparison, is conceptually simpler because... well... the concept is simple. Allocating and de-allocating resources, while tricky at scale, is something for which everyone (including non-programmers) probably have existing mental models for.
Event driven programming, on the other hand, is a slippery high level concept. There are relatively few analogues for it in the "real world", and therefore requires additional mental gymnastics to internalize and understand.
Essentially, we need to 1) follow the execution pattern of event driven code (annoying), while at the same time 2) "visualizing" or conceptualizing a fairly un-natural manner of abstraction.
Deferreds are cool although they have their own set of issues. Mainly, that when you start chaining them there are situations where it can be a bit counterintuitive what is going on. My background is the Deferred from Twisted and Reimplemented in MochiKit.
You really need to read the Deferred implementation if you are going to use it. Otherwise you are asking for trouble long term. Of course, the other issue is that you may run into challenges explaining deferred's to your co-workers. :)
Twisted explored some cool ideas where you basically would write asynchronous code in an interative style using a blend of iterators and generators. Sadly until Javascript has those capabilities in every browser (and not just Firefox) I don't think it is possible.
If v8 implements this, both Firefox and Chrome as well as node.js will have yield, enabling libraries like taskjs [1] to be used.
From taskjs.org:
spawn(function*() {
var data = yield $.ajax(url);
$('#result').html(data);
var status = $('#status').html('Download complete.');
yield status.fadeIn().promise();
yield sleep(2000);
status.fadeOut();
});
Does anyone here have experience with using one of those X-to-JS compilers that do a similar thing but without requiring those Javascript extensions? The only one I hear people talking about a lot is icedcofeescript but I'm not a big cofeescript fan...
The main issue I have with "escaping from callback hell" is that it's a half-truth. Although I don't know much about how the Reactive Framework created by Microsoft works, I know they went well beyond the basics to try to make it all-encompassing coming closer to making it a full-truth.
Just transmitting data back and forth may play well to the strengths of your abstraction. But we have other uses with Timers that should also need such abstractions.
With Timers I have other needs like delaying the execution, resetting the delay countdown, stopping it before it executes it at all (like cancelling it), and finally with an Animation class I needed a way to finish executing a string of events in an instant in order to start a new animation. Also the Animation had other Animation versions at play that could need to be sped up before a new Animation started.
In .NET they seem to have a handy feature that waits the code to run before proceeding that comes into play with their .NET version of the Reactive Framework.
As far as I can tell, it's tough to really solve it. JavaScript doesn't have extra features like .NET does. We are more limited in what we can do. In Dart they have a version of this called Future that has been streamlined recently. As simple as it may seem to be, it comes with other related abstractions called Streams that altogether make it a bit daunting to escape from that hell only to land on the fire outright.
It seems like this problem would be elegantly solved by starting a thread, green thread or coroutine (depending on language) for each task and calling the API functions synchronously from within that. I'm not sure what support JS has for these things.
JS has no support for these things, but you're right, when using languages (like Go) that do support this concept, it is very easy to write elegant code which can block locally without actually blocking the entire application, which allows you to write code that is far cleaner and easier to reason about.
Do you know if there are any good alternatives that are pure-ish JS? I know that same guy worked on tamejs but they don't seem to be working on that anymore...
It seems odd that so many people defend callback nested chains. You end up writing in continuation passing style, when many compilers can do CPS transform for you (Scheme, C#, things like IcedCoffeeScript.
If you have to deal with lots of nested callbacks, it's probably worth to use a CPS transformation like Streamline.js. It makes your code a lot cleaner. As if it was written synchronously.
I recently also tried my hand at promises using the node libs Q and when.
There's a gotcha with the progress handler. If you try to call the progress handler before the progress handler actually gets a chance to attach itself outside the function, it'll never actually fire. Some of the bugs with using promises are rather subtle.
I find LiveScript's back-calls (<-) very elegant. In fact it makes concurrent code very easy to write and comprehend. Combined with async (https://github.com/caolan/async) it is a pure joy.
As for pure JavaScript, dealing with callbacks is definitely not fun.
I just don't understand the issue people are having here. Nested callbacks are a design choice, if you do not want nested callbacks stop writing code with them in it.
Here is a simple example in some Node.js code I've just written (I've never written any before this little project).
Maybe I'm taking the point of the example out of context here, but i worry that promises makes it too easy to write non thought through code like this. Why make two separate http-requests for something that should have been possible to do with one?
Future prediction: Someone will create code similar to the one above but inside a for-loop calling getItem hundreds of times instead of just 2.
You frequently don't control the API's your hitting. I've seen plenty of API's that don't support bulk operations.
I'm not sure I'm convinced if someone doesn't know how to do a bulk operation they're more likely to look because they've got an extra callback or two.
If only someone could come up with an abstraction over all this... We could perhaps call it threads, or co-routines, or such. It could make things so much easier!
Not really better. The advantages of promises show up when you pass promises around, letting other people add their callbacks to it (this way the "callback structure" is implicit from the program flow, just like what happens when you do normal sync programming. Another advantage of promises is that they have some better support for error handling (you don't need to thread the error handler around and you "throw" statements are caught correctly)
Honestly, I didn't like the examples in the article very much. Not only do they fail to show the major advantages promises have, but he also uses named functions everywhere and that is quite silly.
Agh, this is an editing mistake. I re-worded this last minute and reading again next day, it was a mistake.
Really what I meaning to say is that it's much clearer as to what is happening. I was trying to point out that success and error were actually functions on the deferred returned from $.post, not just functions you can pass into $.post (as they once were - hence "circa 2009").
Well, frankly, it looks like you're jumping from one sheep to another. Using deferreds does NOT make more sense, and is actually even more difficult to perceive as it's quite counter-intuitive. MVVM is the way to go for web apps. Actually, anything else would be better than this.
The methods and approaches jQuery provides should not be used as a mainframe to achieve your goal, they should only be used as helpers here and there if ever.
Swizec|13 years ago
They're just continuations, seriously, what's everyone's problem? You define a function, it gets access to the current scope, it defines the rest of the program flow.
If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
Why is this so difficult to people?
pkulak|13 years ago
ef4|13 years ago
A real continuation captures the entire execution context including the current call stack.
Instead what you get in javascript is a stack that's completely meaningless. There's no way to automatically route your exceptions to the real calling context, and there's no automatic way for you to be sure you're seeing all of your callee's exceptions.
If you really want to be sure you'll hear about the ultimate success or failure of an asynchronous call, every single asynchronous step needs to manually install it's own exception handler, trap any exceptions, and pass them back via a failure callback. You're basically doing by hand what the runtime environment is actually supposed to do for you (that being the whole point of exceptions).
rdtsc|13 years ago
If you start seeing do1() do2().. or cb1(), cb2() and so on function that is the "hell" everyone is talking about. Of course you should name your functions better -- but that's not the point. Logically you might not need an extra function but you are forced to add it because of the way your framework forced you to handle IO.
coldtea|13 years ago
That they're badly made continuations, without support from the language.
>If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
At the cost of moving stuff out of where it's invoked, so making code harder to read.
The problem with callback hell is that is pushes the programmer to write FOR the machine, in the way the machine likes it. Those things should be an implementation detail, and in good languages, are.
smosher|13 years ago
I wish `let` was a little different, syntactically. With something closer to what OCaml does, you don't need to surround everything in parens and braces. This would take all the visual grossness out of it (which I think is the biggest problem people really have):
vs. It doesn't look like much, but when you have a number of nested callbacks to define the latter syntax keeps everything on the same level where the former nests just as poorly as defining them in the parameter list. (I also hate seeing '})' anywhere, but not many people seem to care about that so much. It's why I switch to Monaco if I am doing JS.)[Edit: formatting.]
GhotiFish|13 years ago
"there! See how much cleaner that is?"
No, I don't, and this stories solution in particular, is particularly ugly.
etrinh|13 years ago
Let's say you have 3 ajax calls going in parallel. With $.when, you can attach callbacks to arbitrary groupings of those ajax calls (callback1 runs when ajax1 and ajax2 are done, but callback2 runs when ajax1 and ajax3 are done).
I first learned about promises in Trevor Burnham's excellent book Async Javascript (http://pragprog.com/book/tbajs/async-javascript) and it is still the best explanation of promises I've ever read. If you like this article and are interested in reading further about promises or the asynchronous nature of Javascript in general (both for browser and node.js), I highly recommend you check out this book.
ianbishop|13 years ago
daleharvey|13 years ago
I got to write some firefox only code recently and added a delay to a function by just adding
it felt like magic, I dont like generators and would much prefer to see message passing and blocking calls like erlang, but failing that it will be nice to be able to use generators more regularlythejsjunky|13 years ago
Consider that with things like jQuery deferreds I can start a request in one function and pass it around adding handlers in others...something very hard to do with the old CB model. Maybe later on you have some optional action that you only want to occur if a previous action had completed. This sort of thing is easy with deferreds. It makes bundling together disparate actions easier as well as handling series of things as you mention.
Having an abstraction layer in there also has a lot of potential that is (somewhat) untapped. There is a lot of possibility for library authors to add excellent error handling or debugging capabilities in there - things that are currently a bit painful with POC (plain 'ol callbacks).
I agree generators have strong possibilities in this area- I would note thought that they are not an alternative to promises - the two are orthogonal, and in fact they work quite well together. Just take a look at what some other Mozilla folks are doing with http://taskjs.org/ for example.
Sephr|13 years ago
More complex things are also easy to do as well, such as implementing a node.next(event) method, so you can simply do var click = yield document.next("click"); instead of the whole event listener shebang. The feedback example (https://github.com/eligrey/async.js#asking-the-user-for-thei...) shows how much of a difference using yield can be versus callbacks or even promises.
fzzzy|13 years ago
https://github.com/fzzzy/pavel.js
The only caveat is that you do have to write "yield receive('pattern')" instead of just "receive('pattern')".
fzzzy|13 years ago
digisth|13 years ago
https://github.com/caolan/async
I've only used it with node.js, but it's supposed to work in web browsers as well.
It allows you to think a little more procedurally ("waterfall" is especially handy here) while writing CPS code. Very good.
Jare|13 years ago
rbrcurtis|13 years ago
pkulak|13 years ago
doktrin|13 years ago
Memory management, by comparison, is conceptually simpler because... well... the concept is simple. Allocating and de-allocating resources, while tricky at scale, is something for which everyone (including non-programmers) probably have existing mental models for.
Event driven programming, on the other hand, is a slippery high level concept. There are relatively few analogues for it in the "real world", and therefore requires additional mental gymnastics to internalize and understand.
Essentially, we need to 1) follow the execution pattern of event driven code (annoying), while at the same time 2) "visualizing" or conceptualizing a fairly un-natural manner of abstraction.
harshaw|13 years ago
You really need to read the Deferred implementation if you are going to use it. Otherwise you are asking for trouble long term. Of course, the other issue is that you may run into challenges explaining deferred's to your co-workers. :)
Twisted explored some cool ideas where you basically would write asynchronous code in an interative style using a blend of iterators and generators. Sadly until Javascript has those capabilities in every browser (and not just Firefox) I don't think it is possible.
jlgreco|13 years ago
spion|13 years ago
http://code.google.com/p/v8/issues/detail?id=2355
If v8 implements this, both Firefox and Chrome as well as node.js will have yield, enabling libraries like taskjs [1] to be used.
From taskjs.org:
[1]: http://taskjs.org/ufo|13 years ago
estavaro|13 years ago
Just transmitting data back and forth may play well to the strengths of your abstraction. But we have other uses with Timers that should also need such abstractions.
With Timers I have other needs like delaying the execution, resetting the delay countdown, stopping it before it executes it at all (like cancelling it), and finally with an Animation class I needed a way to finish executing a string of events in an instant in order to start a new animation. Also the Animation had other Animation versions at play that could need to be sped up before a new Animation started.
In .NET they seem to have a handy feature that waits the code to run before proceeding that comes into play with their .NET version of the Reactive Framework.
As far as I can tell, it's tough to really solve it. JavaScript doesn't have extra features like .NET does. We are more limited in what we can do. In Dart they have a version of this called Future that has been streamlined recently. As simple as it may seem to be, it comes with other related abstractions called Streams that altogether make it a bit daunting to escape from that hell only to land on the fire outright.
tomlu|13 years ago
dons|13 years ago
Simon Marlow captured this well: http://stackoverflow.com/a/3858684/83805
"the abstractions that threads provide are essential for making server code easier to get right, and more robust"
georgemcbay|13 years ago
epochwolf|13 years ago
jart|13 years ago
ufo|13 years ago
lucian1900|13 years ago
jj2|13 years ago
https://github.com/Sage/streamlinejs http://en.wikipedia.org/wiki/Continuation-passing_style
iamwil|13 years ago
There's a gotcha with the progress handler. If you try to call the progress handler before the progress handler actually gets a chance to attach itself outside the function, it'll never actually fire. Some of the bugs with using promises are rather subtle.
cwiz|13 years ago
As for pure JavaScript, dealing with callbacks is definitely not fun.
Offler|13 years ago
Here is a simple example in some Node.js code I've just written (I've never written any before this little project).
https://github.com/Offler/node-bundler/blob/master/lib/bundl...
As you can see you just need to write in an OO manner and use function.bind(this) and Bob's your uncle. Really don't get the difficulty.
Too|13 years ago
Future prediction: Someone will create code similar to the one above but inside a for-loop calling getItem hundreds of times instead of just 2.
ajanuary|13 years ago
I'm not sure I'm convinced if someone doesn't know how to do a bulk operation they're more likely to look because they've got an extra callback or two.
moe|13 years ago
rorrr|13 years ago
ufo|13 years ago
Honestly, I didn't like the examples in the article very much. Not only do they fail to show the major advantages promises have, but he also uses named functions everywhere and that is quite silly.
ianbishop|13 years ago
Really what I meaning to say is that it's much clearer as to what is happening. I was trying to point out that success and error were actually functions on the deferred returned from $.post, not just functions you can pass into $.post (as they once were - hence "circa 2009").
bestest|13 years ago
The methods and approaches jQuery provides should not be used as a mainframe to achieve your goal, they should only be used as helpers here and there if ever.
lucian1900|13 years ago