top | item 4732924

Escape from Callback Hell: Callbacks are the modern goto

247 points| wheatBread | 13 years ago |elm-lang.org | reply

147 comments

order
[+] tikhonj|13 years ago|reply
If you don't want to use another language and compile down to JavaScript--which is what Elm offers--there are some interesting options that are just JavaScript libraries.

The one I've personally played with is called Arrowlets[1], which introduces a control structure called an arrow that lets you abstract over callbacks and event handling (among other things). Using that style of programming can significantly simplify some fairly common tasks in JavaScript; the drag-and-drop demo on their site is a good motivating example. However, unless you are already familiar with functional programming and arrows, you should probably read some background before diving into the examples.

[1]: http://www.cs.umd.edu/projects/PL/arrowlets/

Another interesting option I've toyed with is RX.js[2]. This is a JavaScript version of C#'s Reactive Extentions (RX). If you are familiar with Linq, then this library's style should seem natural immediately. The core idea here is to abstract over events as streams that can be composed and manipulated conveniently.

[2]: http://rxjs.wikidot.com/

If you don't mind using a different language, but want something that mostly looks like JavaScript, another option is FlapJax[3]. I haven't tried it myself, but I've certainly heard good things about it.

[3]: http://www.flapjax-lang.org/

There are probably more options in the same vein that I forgot or don't know about. However, I think these three are a good starting point and could help clean up much of your event-driven JavaScript code in the short term.

Of course, if you are willing to use a language radically different from JavaScript, then Elm is a great option. Once you get used to functional languages with good type systems, there is really no going back ;). The syntax is also simpler and more minimalistic than JavaScript's, which leads to more readable code.

[+] lautis|13 years ago|reply
Bacon.js[1] is another functional reactive programming library for JavaScript. Many similar concepts to RX.js, but the library is much smaller. While Bacon.js might not be the most mature library around, I've used it successfully on (almost) daily basis for few months.

[1] https://github.com/raimohanska/bacon.js

[+] morsch|13 years ago|reply
It would be interesting to see the different libraries/languages side by side solving a benchmark problem like the drag and drop example.
[+] magicalist|13 years ago|reply
Callback hell is certainly a real thing, but that Javascript snippet is a poor example for a goto comparison, since it's pretty much as linear as you can get.

The problems with Javascript and callbacks are usually (in reverse importance): noisy verbosity (all those "function()"s), the deeper and deeper indentations, and then ensuring execution order on interdependent async steps while keeping it readable. In the blog post's example, you pretty much of a serial chain of dependent steps, and the only thing really wrong with it is that it's just ugly and approaching unreadable (syntax highlighting will help quite a bit, though).

I think most people heavily involved with Javascript recognize those problems, though. Promises/deferreds have entered mainstream js usage. They can be somewhat confusing for newcomers, but several libraries can help, as others have pointed out. Language support is evolving: "let" as an option for more control over scoping, the arrow syntax for simpler function expressions, yield for shallow continuations, etc. These will in turn feed back into making libraries smaller and easier to use (I'm really looking forward to when I can use http://taskjs.org/ for all my async needs. Combined with the arrow syntax, I feel like I can pretty much always avoid a callback mess and retain clarity of (high-level) flow at a glance).

This isn't a knock on elm (this article is the extent of my knowledge of it), and it isn't a dismissal of the problem, but it isn't clear to me from this article what is broken in JS that is fixed in elm. In other words, this could be another tutorial on promises in Javascript and make the same points about excessive callbacks being poor coding style and bad news for readability and maintainability.

Syntax that makes clear code the lowest energy state is a feature, but (if we limit our discussion to callbacks) in JS it's partly solved, partly being worked on, and it's not clear to me yet what the energy differential is in typical elm usage between this code and the nasty spaghetti code you can always write if you try.

[+] ender7|13 years ago|reply
Personally, any solution to callback hell will also need to be a language that supports returning multiple values. The following convention is simply too useful to me:

  foo.doSomethingAsync(function(err, result) {
    if (err) {
      ...
    }
    ...
  });
You can obviously accomplish this with exceptions, but then you have a million try/catch blocks floating around all over the place and the code becomes even harder to read (and more verbose to boot).
[+] pufuwozu|13 years ago|reply
There's a way without exceptions. Use an ErrorT[Promise[A]] monad. I've done this in Scala before and it is so much simpler than the JavaScript convention you pointed out.

It allows you to write code like this:

    val query = "Brian"
    val result = for {
      db <- connectToDatabase
      user <- userFromDatabase(db, query)
      friends <- friendsFromDatabase(db, user.friends)
    } yield friends
Whenever one of the functions above returns an error, the whole expression is that error. The outcome is either an error (if there was any) or the end value of Set[User]. No need to manually handle an error until you get to the end result.
[+] zoowar|13 years ago|reply
I've always seen this as an odd convention. Why not

if (typeof result === Error) {

[+] zurn|13 years ago|reply

  foo.doSomethingAsync(function(result) {
    ...
  }, function(err) {
    ...
  });
[+] webjprgm|13 years ago|reply
The "callback hell" example is a rather tame one, since the code is readable in a single location just with some nesting. So when I see the FRP solution which is the same amount of code I'm not certain that in a complex example this actually solves the problem. You can still have lift statements scattered around a program just like you can have callbacks to various functions scattered around.

The solution to GOTO was to remove it and replace it with a few control structure forms that enforced locality. I remember converting someone's GOTO-laced code once and basically everything could be re-written with some clever use of do-while with break statements and an occasional flag variable. do-while, while, for, etc. replace GOTO in 99% of cases and enforce the desired code locality for readability.

So what syntactical structure could enforce locality of time-connected variables?

E.g. some idea like this:

    data, err <- $.ajax(requestUrl1)
    if( err ) {
      console.log(err)
      return
    }
    data2, err <- $.ajax(makeRequestUrl2(data))
Where the <- syntax could be like an = statement but say that the assignment and all following statements are placed in a callback.
[+] ufo|13 years ago|reply
I think there are a couple of async Javascript dialects that already do this:

http://tamejs.org/

The main advantage tat not many people see is that you get backwards compatibility with sync code (if statements, while loops, etc) basically for "free".

[+] btilly|13 years ago|reply
More than 99% of cases.

If you have functions, loops, and nested loop control, in 100% of cases you can replace GOTO code with code that is equally efficient. With flag variables and if checks, you can likewise always replace the code, BUT efficiency and verbosity suffers because of the need to check the flag variable repeatedly in the loop.

As for your syntax suggestion, I am reminded of http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html which manages to use a preprocessor trick to generate working coroutines in C. (This trick is actually used in PuTTY.) I also think that the strategy that you describe would work well enough for my tastes.

[+] debacle|13 years ago|reply
Callbacks are different than gotos in that they are aren't even remotely close to gotos.

With a callback, you can get into 'callback hell,' however the root cause of that is that you probably don't understand the nuances of properly architecting a solution that involves the power of first-class functions.

JavaScript is nice because the scoping of the callback is easily controllable through your invocation method, and if you've created a good object model then it's relatively easy to maintain an understandable state.

When you explicitly define callbacks like in the examples, you're tightly coupling the response handlers to their requests, which is a relatively poor implementation and will bite you in the ass later on.

[+] pufuwozu|13 years ago|reply
Callbacks are different than gotos in that they are aren't even remotely close to gotos.

The analogy is that callbacks create non-linear control flow. Using a monadic syntax like in Roy, we can easily have callbacks without having them look non-linear:

    let deferred = {
      return: \x ->
        let d = $.Deferred ()
        d.resolve x
        d.promise ()
      bind: \x f -> x.pipe f
    }

    let v = do deferred
      hello <- $.ajax 'examples/helloworld.roy'
      alias <- $.ajax 'examples/alias.roy'
      return (hello ++ alias)

    v.done console.log
Which compiles into continuation passing:

    var deferred = {
        "return": function(x) {
            var d = $.Deferred();
            d.resolve(x);
            return d.promise();
        },
        "bind": function(x, f) {
            return x.pipe(f);
        }
    };
    var v = (function(){
        var __monad__ = deferred;
        return __monad__.bind($.ajax('examples/helloworld.roy'), function(hello) {
            return __monad__.bind($.ajax('examples/alias.roy'), function(alias) {
                return __monad__.return((hello + alias));
            });
        });
    })();
    v.done(console.log);
Anyway, continuations (with call/cc) are definitely a controlled form of goto. Take a look at an example from Paul Graham:

http://lib.store.yahoo.net/lib/paulgraham/cint.lisp

    ((call/cc
      (lambda (goto)
        (letrec ((start
                  (lambda ()
                    (print "start")
                    (goto next)))
                 (froz
                  (lambda ()
                    (print "froz")
                    (goto last)))
                 (next
                  (lambda ()
                    (print "next")
                    (goto froz)))
                 (last
                  (lambda ()
                    (print "last")
                    (+ 3 4))))
          start))))
[+] barrkel|13 years ago|reply
Along with a trampoline, callbacks can be used to implement continuation passing style (CPS), which is more or less a fully generalized goto - not just static goto, but computed goto. CPS is sometimes called "goto with parameters".
[+] seanalltogether|13 years ago|reply
"When you explicitly define callbacks like in the examples, you're tightly coupling the response handlers to their requests, which is a relatively poor implementation and will bite you in the ass later on."

I agree, if anything this is an argument about notification vs delegation. Callbacks should be used when you need to delegate a feature, like when an array object needs to call outside its scope to ask for a sorting function. Notifications should be used when the calling object doesn't care who handles the information, like in the case of an asynchronous load, and in a proper notification environment you wouldn't have the spagetti code this blog is illustrating.

[+] Sephr|13 years ago|reply
One way to escape callback hell is to use a async.js (https://github.com/eligrey/async.js), which uses yield to abstract away callbacks. It's Firefox-only (JS 1.7+) though, but that can probably be resolved by using a JS parser and replacing every yield with JS 1.5 callbacks.

Full disclosure: I wrote async.js.

[+] wyuenho|13 years ago|reply
The problem is not callback, the problem is that callbacks exists in Javascript.

Callbacks themselves, when used wisely, can often enhance code readability, hell LISP has had function references since forever, but I think the most complain about callbacks are actually complains about callbacks in noisy languages, mostly likely languages with noisy syntaxes like Javascript and Java. When read that way, the disgust towards callbacks do seem to have merits. As the author has pointed out, the 2 getPhoto() functions at the end express and do exactly the same things, but obviously the CoffeeScript version reads better.

Callbacks have been around a long time and I've never heard of people complain as much about them as people have for Javascript and I conjecture the reasons are as follows:

1) There's no named parameters (keyword arguments) in Javascript, so people pass around objects literals into functions to emulate them. 2) Making lambdas in JS is too easy, but the syntax is too noisy. 3) Oh so many aliasing of this; 4) Self-chainable JS libraries like jQuery makes the style of calling multiple functions too easy. But lines can only go to long before becoming unwieldy, so people tend to indent chained method calls multiple times. 5) No modules and global namespace pollution is frown upon, so people are hesitant to flatten deeply nested callback chains. 6) There are a dozen ways to make a JS class and/or object depending on frameworks, and they are not at all compatible.

All of these "features" coagulate in JS into giant blobs of snot like this:

  <script>
    $(document).ready(function() {
      var $main = $("#main");
      $main.
        hide().
        click(function(e) {
          $.ajax({
            type: "POST",
            dataType: "json",
            contentType: "application/json",
            success: function(data, textStatus, jqXHR) {
               data['rows'].forEach(function(line) {
                   $main.append($("<p>", {
                      className: "row"
                   }).append(line));
               });
            }
          });
        }).
        show();
    });
  </script>
Words for the wise, when you see a shiny new jQuery plugin, stop, think for 3 minutes, and then put it inside a Backbone View or whatever your favorite framework is other than jQuery*. If you don't know anything other than jQuery, now is probably the best time to learn a few.
[+] herge|13 years ago|reply
This reminds me of the pain of dealing with python's twisted library, albeit before inline callbacks were implemented.

Inline callbacks as implemented in python can make asynchronous code a lot easier to read: http://hackedbellini.org/development/writing-asynchronous-py...

[+] pjscott|13 years ago|reply
And using something like Eventlet or Gevent can make asynchronous code downright pleasant. Of the two, Eventlet has the better docs, and both are easy to get started with, and stable enough for production use:

http://eventlet.net/doc/

Give it a try! You almost definitely won't regret it!

[+] peterbe|13 years ago|reply
Isn't `yield` the solution to all the problems? It makes things responsive and avoids the callbacks entirely.

For example: http://www.tornadoweb.org/documentation/gen.html

[+] masklinn|13 years ago|reply
It is a solution, kind of, but annoying on a few accounts:

0. it plays hell with threadlocals-as-dynamic-scoping, which is the only way most "modern" languages permit dynamically scoped variables

1. it needs to be correctly passed along to callers, and given it's usually used in dynamically typed languages there's a high risk of forgetting and dropping a yield

2. yield being also used for iteration, it can be confusing to keep them straight.

It's definitely a better solution than callback hell though. An other approach is runtime support as in gevent, where the "yielding" is done by the library/IO code and invisible to the caller. The final two I know of are baking lightweight concurrency and communications into the language itself (Erlang, and to a lower extent Go and Rust) or monadic systems (Haskell)

[+] ccleve|13 years ago|reply
You've got to ask, why is async programming used at all? The reason is twofold: first, the C10K problem, where too many threads kill performance, and second, sometimes you want to have multiple tasks run in parallel.

There are fairly simple syntactical solutions to both problems.

  result = doSomeAsyncTask()
  result.yield() // drops the thread, picks it up on response
  // do stuff with result here
This magic yield() doesn't exist (to my knowledge), but if it did, it would preserve linear code and also solve the C10K problem.

You could have similar code to solve the multiple task problem:

  result0 = doSomeAsyncTask0();
  result1 = doSomeAsyncTask1();

  while (result = getNextCompletedTask(result0, result1)) {
    // do something to result
  }

A Future in Java does something like this, but it doesn't drop threads.
[+] jblow|13 years ago|reply
Callback Hell is certainly a real thing. I decided 12 years ago that I would never use callbacks if I could avoid it (the only wai you can't avoid it is if an API forces you to use them); I have never looked back. Regular, simple, straightforward imperative flow control is a very powerful thing, and any time you give it up or make it more squishy and indirect, you had better be getting something big in return. Usually you aren't.

That said, what the article proposes as a solution is bananas. You don't need to do crazy functional acronym things; just don't use callbacks. Good C/C++ programmers in the field where I work (video games) do this all the time. It's not hard except that it requires a little bit of discipline toward simplicity (which is not something exhibited by this article!)

[+] tactics|13 years ago|reply
Actually, you're wrong. This is bananas (based on very closely related FPR concept)

http://www.haskell.org/haskellwiki/Reactive-banana

Secondly, you come off sounding defensive and ignorant. This is a new programming paradigm. Hopefully it will give people new ways to approach the same difficult problems. (And I really hope you believe GUIs are inherently difficult...)

No one is twisting your arm to learn FPR. If callbacks work for you in your job, then stick with what works.

[+] Quiark|13 years ago|reply
Ok, I understand that you don't use callbacks. But you forgot to mention what you use instead of them, in situations such as described in the article. Polling?
[+] pkaler|13 years ago|reply
Boost bind and fast delegates do certainly suck. (I used to work in games in a past lifetime.) But this is most certainly a C/C++ problem.

In Objective-C, the @protocol keyword gives the language first class delegation and works really, really well. More details here: http://developer.apple.com/library/ios/#documentation/Cocoa/...

With respect to the original article, he's talking about callbacks with respect to Node.js. That's not a callback issue. Async is unnatural for the mind to grasp. What did he expect?

[+] chipsy|13 years ago|reply
What do you do for networking, though? That seems to be the motivation to introduce callbacky code in web programming.

(FWIW my own architectures tend to turn callbacks into queues and polling.)

[+] Jacob4u2|13 years ago|reply
The author offers an alternative that would require a change to the language. Callbacks and their use in "callback hell" are a little different than use of "goto"; "goto" appears to have an obvious alternative that was more logical to use already implemented in the language. For javascript, there is none of the nice syntactic sugar (reminds me a lot of C# recent async changes) that the author suggests and is not even being proposed for ECMA 6.

I agree it would be nice to have that stuff, and that callbacks can get a little hairy, but they are the best solution available at present. Shall we stop developing applications in the mean time while the language catches up, or even worse, browsers actually consistently implement the changes?

[+] dllthomas|13 years ago|reply
No, we should stop writing javascript and start writing elm - a language that compiles to javascript, works like described in that page, and is the subject generally of the site hosting the article.

At least, that's what the article is saying. There are a few interesting things on the horizon there, but I've been watching elm with some interest.

[+] tactics|13 years ago|reply
If you had a program back then full of gotos and an assembly language that didn't support structured blocks, your problem was just as bad.

At its core, FPR only requires higher-order functions to work. (And nearly every modern language supports them to some degree).

The things that Elm provides are additional niceties:

* A type system with parametry polymorphism (aka generics) helps you spot otherwise nasty runtime errors ("expected a function, got a signal").

* Abstract data types - The only way to create a signal is through the API. The only thing you can do with a signal is pass it around and feed it back into the API.

* Language purity - This one is probably the hardest sell for average languages, since every modern language (save Haskell) allows for unrestricted side-effects. However, as long as you don't bypass the API and update the UI directly, you don't actually NEED purity.

The nice thing about Elm is that it compiles directly to Javascript. You can integrate it into new pages on your existing site without giving up anything. I think the language -- and more generally FPR as a basic tool in your toolkit -- has a lot of potential in the future.

[+] benregenspan|13 years ago|reply
There are alternatives that don't require extending the language or having to work directly with a mess of nested callbacks. A bunch of flow-control libraries here: http://dailyjs.com/2012/02/20/new-flow-control-libraries/

This is definitely more hairy to use than if the language supported it natively, but not so bad, and can be implemented in a very lightweight way (this approach http://daemon.co.za/2012/04/simple-async-with-only-underscor... is a few lines of code on top of Underscore.js which a lot of sites are using already).

(PS: sorry if OP's article already mentioned this stuff, I can't load it at the moment)

[+] wglb|13 years ago|reply
This kind of confuses two important ideas, both discussed by Dijkstra.

The most popular was his article about gotos.

Another idea in his writings was that time-dependent programming was dangerous. He was talking about interrupt based programming specifically, and also addressed the common practice of some hardware to have asynchronous IO. You would start an IO operation, and go on and do other things, come back later and see the values there.

So these two things are not alike. They both cause confusion about what the program is doing, but they are not "like" each other.

To be a better programmer, it is good to read Dijkstra. It is really all about avoiding errors in programming.

[+] darwinGod|13 years ago|reply
As someone who writes C code for a distributed system that uses event-driven callbacks ( Zscaler) (yes,the binding is at compile time), I was aghast when I saw goto's in the codebase. I mean,I believed programmers were indoctrinated with " using goto = goto hell". I have realized that if used smartly,goto's cause no problem-say in error handling. I can confidently say I have not seen a single bug because of improper usage of goto in the last 1.7 years. And we do a lot of interesting things in C,including talking to a Postgres database,having a messaging protocol layer,doing shared memory manipulation etc.
[+] grimtrigger|13 years ago|reply
One thing I'd like to see from languages that compile to js: Some kind of evidence that output readability is a concern. You can make some beautiful abstractions, but if I can't debug it when things go wrong, then there's no way I would use it.

Not making any comments about Elm's output, but the author clearly doesn't consider it a priority in the post.

[+] Evbn|13 years ago|reply
But how? Does your C code compile to readable assembly? Does Haskell compile to readable C? The high level language is created because the lowlevel language is unreadable when solving problems the high level language solves.
[+] whatgoodisaroad|13 years ago|reply
This seems like a bad analogy. Dijkstra's paper was in favor of "structured programming",and the problem was that goto was too-unstructured. If anything, callbacks are excessively structured.

Also, why is nonlinear code a bad thing? If the program behavior should be nonlinear, then neither should the code.

[+] Evbn|13 years ago|reply
Because people have trouble reasoning mon-linearly.
[+] chubbard|13 years ago|reply
Two observations. First, great how do we debug it? How can we see our signals between each step? How about beyond simple print/logging?

And two, I like his contrast between async vs synchronous flows, and recognizing synchronous style programming has many benefits that CPS doesn't. However, I think even this style still hasn't solved the bigger problem with asynchronous style programming. The ability to reuse it easily. In synchronous style programming I can reuse code and add to that block by calling the method, then after that method is done add my code.

   ... my code before ...
   var result = someMethod()
   ... my code after ...
It's just that simple with synchronous style. With async style the author has to provide you a hook to hook onto the end or beginning of this flow (adding a callback param, returning a promise, etc). I think even with using signals you have the same issue. Without explicit hooks you can't hook more code onto it like you can with good old fashion synchronous programming. Not to mention error control flow is turned upside down too.

I'm intrigued by the ideas of signals over callbacks, but I don't know if they fix enough problems with callbacks yet.

[+] graue|13 years ago|reply
The following is a dead comment by seanmcdirmid... reposting since it seems perfectly legitimate and I have no idea why this would be voted down. (Glitch?)

Debugging is one of the problems with FRP/signals, or functional code in general. No one has come up with a good dataflow debugger yet, and it might not even be viable. The best you can do is interpose "probes" on your code like you would take measurements with an oscilloscope. Disclosure: I did my dissertation on signals (object-oriented ones to be precise), and am a bit disillusioned with it.

On the other hand, the argument from the declarative community is that you don't need to debug your code.

A better alternative to FRP/signals might be immediate mode user interfaces. Since they are conceptually called on every frame, you get the benefits of FRP while still being able to debug in the old way. On the other hand, they are quite inefficient, though I think we could play some tricks with technology to make them better (memoize, refresh blocks of computations only as needed via dependency tracing).

[+] jawns|13 years ago|reply
So, this Functional Reactive Programming stuff compiles to Javascript, right?

Is the resultant Javascript just a bunch of nested callbacks, as in the example the blog post uses to illustrate spaghetti code?

[+] eblume|13 years ago|reply
The 'big problem' with callbacks isn't the callbacks themselves - it's the resulting difficulty to the developer to keep track of the flow of control.

This is analogous to goto's - programming with goto is a nightmare, but all code 'compiles to' some sort of immediate form using goto's. The goto is not the problem, programming with goto is.