Kal – a clean JavaScript alternative without callbacks

[rybosome]

It's remarkable how similar this code looks to the monadic way of doing it.

For this Kal code...

    task getUserFriends(userName)
      wait for user from db.users.findOne {name:userName}
      wait for friends from db.friends.find {userId:user.id}
      if user.type is 'power user'
        for parallel friend in friends
          wait for friendsOfFriend from db.friends.find friend
          for newFriend in friendsOfFriend
            friends.push newFriend unless newFriend in friends
      return friends

Here's a pseudo-ish implementation in pseudo-ish Haskell.

  getUserFriends userName = do
    user <- findUser userName
    friends <- findFriends user
    if (User.type user) == "power user"
      then friends ++ parMap rdeepseq $ (getSecondNodes friends) friends
      else friends
		
  getSecondNodes firstNodes friend = do
    secondNodes <- findFriends friend
    diff firstNodes secondNodes

I like the concept of Kal, and I look forward to seeing what people do with it. Regarding syntax, I have to admit that I share the opinions of a few others in terms of preferring symbols over so many keywords, but that's a minor nitpick. Great job!

[rzimmerman]

Thanks! I had not thought to make the comparison to Haskell and monads. That's pretty interesting.

A lot of people prefer symbols over keywords, and I think that's mainly a readability issue. I personally prefer more keywords with good syntax highlighting, so that's what I went with (and why I almost immediately made a .tmbundle).

[GeZe]

If you are looking for a syntax closer to Haskell's, you can take a look at LiveScript (http://livescript.net/)

eg.

    getUserFriends = (userName) ->
      user <- findUser userName
      friends <- findFriends user
      ...

Specifically, the section on "backcalls": http://livescript.net/#backcalls

[pcj]

This is beautiful work! I see small shades of inspiration from the .Net TPL[1] and the more recent async-await paradigm[2] in C#. But, this is way more concise and beautiful implementation :-)

[1] - http://msdn.microsoft.com/en-us/library/vstudio/system.threa...

[2] - http://msdn.microsoft.com/en-us/library/vstudio/hh191443.asp... and http://msdn.microsoft.com/en-us/library/vstudio/hh156528.asp...

[eru]

And, of course, the Haskell do-notation gets de-sugared into tons of call-backs by the compiler.

[rzimmerman]

This is a project I've been working on in my spare time for a while now and I've finally decided to throw it out there and get some feedback. Do you think the 'wait for' callback syntax is useful? What big features do you think are missing?

[ricardobeat]

The async handlers looks great, specially parallel and series iterators. I'm not sold on

    wait for friends from db.friends.find()
    //vs
    friends = await db.friends.find()

it's a bit hard to follow despite the natural language.

Is this a fork of the CoffeeScript compiler? Will it benefit from upstream changes/features?

[msoad]

Looks pretty good! I like it. To be honest I will not use it in my projects. I prefer using ES6 syntax for this problem when it's available.

[skybrian]

I'd be more interested in performance than features. If you use tasks when they're not needed, how much more JavaScript code does it take and how fast does it run? Also, what do stack traces look like?

[ilaksh]

Are you aware of the ToffeeScript project? Can you make them a website and start promoting it? Because you are missing all if the features in ToffeeScript/CoffeeScripy and ToffesScript needs a shiny website because that's how people judge languages now is how shiny their website is apparently based on the fact that no one seems to care about ToffeeScript. BTW no I'm not the author of ToffeeScript.

[WayneDB]

This is a fantastic spare time project. At the very least, it will be a beautiful feather in your cap :)

[aufreak3]

(Don't mean to divert this thread from Kal, but this is on the topic of "callbacks are bad" etc.)

For some time now, I've been accumulating async patterns I've needed when writing JS code in my monadic IO.js library[1]. The aim of IO.js is to provide higher levels of thinking about sequences of actions without worrying about whether they are async, and with a rich set of error management tools (for ex, exceptions are turned into recoverable conditions in IO.js).

The crux of the "callback hell" problem is, contrary to what many have claimed, is not the nesting that results when the callbacks need to be called in temporal order. That much is straightforward to deal with. The "hell" rears its head when you need to coordinate multiple such sequences that are running concurrently. The composable actions you get from a monadic treatment combined with CSP-style channels, are an expressive framework to build abstractions on (tldr - Haskell's implementation is awesome!).

For illustration, the framework in IO.js is flexible enough to implement a node.js web server that can express PG's "Arc challenge" concisely (though that challenge is practically obsolete). Take a look at [2].

For a simpler example, `IO.trace` is a straight forward way to generate a trace dump to console of a sequence of asynchronous actions. You don't need to "enable trace" for an entire app. You can choose to trace only a particular sequence. This is pretty neat when debugging. Stack traces are useless when dealing with async processes anyway.

[1]: https://github.com/srikumarks/IO.js [2]: https://github.com/srikumarks/IO.js/blob/master/examples/arc...

[molf]

Excellent work. Especially useful since it appears to throw exceptions. That means it's a step up from Iced Coffeescript.

The syntax is a little verbose and might be shortened to something like C#: `user = await db.users.findOne {name:userName}`. Would be a lot more clear to me.

[crazygringo]

As far as I can tell from reading the readme, it does NOT support throwing exceptions from within callbacks.

Because, of course, there's no way to technically do that in anything that is JavaScript at its core -- at least as far as I'm aware of. Correct me if I'm wrong?

It says "This includes error handling via callbacks." -- not error handling via exceptions. Elsewhere it says "Any errors reported by fs.readFile (returned via callback) will be thrown automatically.", so maybe it converts certain callback error functions to exceptions? But in any case, that's not the same as throwing an exception inside of a callback, and having it "bubble up". Unless I'm misunderstanding (which would be great!).

[rzimmerman]

Definitely play with the exception handling. Latent bugs aside, asynchronous statements (wait fors) work in any crazy horrible nested combination of ifs, fors (parallel or series), and trys that you can think of. Take a look at the JS output with:

kal -o output.js -f beautify input.kal

to see the sausages being made. The http server demo is a good example (https://github.com/rzimmerman/kal/blob/master/examples/async...)

[kev6168]

I suggest to use ..= (or ...=) in place of the keywords 'wait for', and 'from'. Less words, and '...' traditionally means 'to be continued', or 'wait for a while'.

For this Kal code:

  task getUserFriends (userName)
    wait for user from db.users.findOne {name:userName}
    wait for friends from db.friends.find {userId:user.id}
    return friends

would become:

  task getUserFriends (userName)
    user ..= db.users.findOne {name:userName}
    friends ..= db.friends.find {userId:user.id}
    return friends

[10098]

The title is kinda funny to the russian ear because kal means feces in Russian :)

[rzimmerman]

That is really unfortunate...

I chose it because it in Hebrew it roughly means something like easy/simple/BASIC.

edit: More importantly, does that make you more or less likely to use it?

[pjmlp]

Well, even worse is the Portuguese name Rui.

I was once told it also sounds like something bad in Russian, so it can be a bit inconvenient introducing anyone with such name in Russian if not aware of this issue.

[mjackson]

For the curious, the equivalent example from the OP using promises[1]:

  function getUserFriends(userName) {
    return db.users.findOne({ name: userName }).then(function (user) {
      return db.friends.find({ userId: user.id });
    });
  }

[1]: http://promises-aplus.github.io/promises-spec/

[tlrobinson]

Promises + CoffeeScript really hits the sweet spot for me:

    getUserFriends = (userName) ->
      db.users.findOne(name: userName).then (user) ->
        db.friends.find userId: user.id

Or prezjordan's example:

    getUserFriends = (userName) ->
      db.users.findOne(name: userName)
        .then (user) ->
          db.friends.find userId: user.id
        .then(function (friends) ->
          somethingAsync(friends)

The concise function syntax and implicit "return" really helps.

If that's not good enough you can implement higher level control flow abstractions much more nicely than with raw callbacks, e.x. https://github.com/tlrobinson/q-step

    getUserFriends = (userName) ->
      QStep(
        ->
          db.users.findOne name: userName
        (user) ->
          db.friends.find userId: user.id
        (friends) ->
          somethingAsync friends
      )

[prezjordan]

It's more obvious if you have multiple levels.

  function getUserFriends(userName) {
    return db.users.findOne({ name: userName })
      .then(function (user) {
        return db.friends.find({ userId: user.id });
      })
      .then(function (friends) {
        return somethingAsync(friends);
      });
  }

[rzimmerman]

Thanks for the comparison, I should probably add promise syntax to the landing page to cover more bases.

I think a useful case would be the trickier stuff where Kal really shines. For example, doing async stuff inside of loops, conditional calls (where some paths need async waits and others don't) and error handling. I also think the Kal syntax might appeal to people who are less experienced with JavaScript and it's more eclectic features.

[pkorzeniewski]

Promises are a completely sufficient solution to handle asynchronous code in my opinion and there are already several great libraries for this in JavaScript. I don't get it why would I want to switch to a new language just because of that one feature.

[tuxracer]

https://github.com/kriskowal/q solves this today without needing to cross compile. http://koush.com/post/yield-await-v8 will address this natively however unlike Q or Kal you can't start using it today.

[ricardobeat]

There is no need for hacks, just run node with the `--harmony` flag to get yield support today.

Q is a huge, monster library. It's 2x the size of caolan/async, which is already bloated. I personally think it's better to use https://github.com/mbostock/queue or another small, understandable library.

[rzimmerman]

Q and ECMAScript 6 solve the callback issue, but I think people might like the built in async iterators and error handling.

[ufo]

Does anyone know if there is a JS dialect out there that adds support for callback-less async code without also adding a bunch of other coffeescripty syntax changes at the same time?

[neilk]

Yes, it's called "JavaScript". Specifically, combining generators with promises.

http://blog.alexmaccaw.com/how-yield-will-transform-node

Definitely not as clean as a whole new language, but you can program like this today, at least in bleeding edge node and some recent browsers.

[simonster]

That would be TameJS: https://github.com/maxtaco/tamejs

[mjw]

Try http://www.neilmix.com/narrativejs/doc/ -- "a small extension to the JavaScript language that enables blocking capabilities for asynchronous event callbacks".

[gfxmonk]

StratifiedJS allows you to write callback-less async code, including full support for exceptions thrown from async code (stacktraces and all):

http://onilabs.com/stratifiedjs (disclosure: I work at Oni Labs)

It does extend JS with some additional syntax as well (new concurrency constructs as well as some syntactic sugar), but it doesn't alter existing JS syntax.

[jay_kyburz]

rzimmerman I like it! and would consider using it for my next big project.

If I were you I would make a slight course correction follow in the footsteps of Python and Go and have one and only one correct way to do something.

When you say use two spaces, but you can use more if you want, make it an error to use more.

Pick the correct way to call a function. Pick the correct way to declare a function.

Keep up the good work.

[rzimmerman]

Thanks! I wouldn't call it stable or production ready by any means, just as a disclaimer.

[dogles]

I am a big fan of languages trying to solve callback hell. I use a library in Haxe that allows for similar syntax: https://github.com/proletariatgames/haxe-continuation. Haxe compiles to Javascript and is great for node programming...but it has the added benefit of also compiling to c++, php, java, c#, etc., etc. ;)

[LordHumungous]

JS without callbacks... so JS without the best part of JS?

[rzimmerman]

It's admittedly a bit link-baity. The async structures in Kal do compile to callbacks in JS.

[gnaritas]

How exactly are callbacks the best part of JS? Callback aren't a language feature, they're a pattern of doing async code with anonymous functions. Anonymous functions are used for far more than just async code and cannot be replaced with an await keyword which only replaces the need to use callback for async code.

[ufo]

Dont confuse closures with callbacks. Asynchronous non blocking code is the nice thing and callbacks are the price you have to pay to use that in JS,

[anuragramdasan]

Of the tonnes of Javascript libraries that are unloaded on HN regularly, this is one of the more fascinating ones. Absolutely love the syntax. Great job!

[TheZenPsycho]

So instead of encouraging you to actually fix the bad nested callback code, you can dress it over with fancy syntax.

Well done on making a whole language though. It's pretty challenging and I hope you do well.

Aside: This seems very similar to narrativeJS in concept http://www.neilmix.com/narrativejs/doc/

[harlanji]

I came across Galaxy yesterday, linked to from the callbacks/goto discussion. It's an NPM module that gives C# async/await semantics to JS/es6 code (of course it requires the harmony VM). It simply toggles function*/function on things, such as required modules, and provides some extra helpers... the author gives a really nice walkthrough:

https://github.com/bjouhier/galaxy

Galaxy does not claim to get rid of callbacks, though... only to help make writing JS/es6 with async/await semantics easier. I've yet to try it out, but the instructions seem clear.

[pyalot2]

How many times do I have to say this. Generator/Yields and equivalent constructs don't actually solve the callback problem, they exacerbate it. That is, because you only get to direct one call-level down, but, that's now how you'd code. You'd have some perfectly synchronous code until it hits some infrastructure I/O part some N call layers down, and then now what? You're gonna convert everything to yields? Got any idea how messy that is?

Javascript needs proper co-routines. Nothing short of co-routines will solve the issue. Please lookup microthreads, greenlets, fibres, etc.

[ronreiter]

What's up with everyone trying to re-invent coroutines? I personally think people should upgrade JavaScript instead of inventing new languages all the time, since it creates fragmentation.

[tlb]

I'm puzzled by the bootstrapping process. It's written in Kal itself, and you distributed compiled code as JS. Do you have a way of re-boostrapping from only the source in github?

[rzimmerman]

The only way to re-bootstrap is to download a compiled (JS) release from npm, then use that to compile the .kal source in the git repository. Then you can use your newly compiled version of kal to recompile itself. This is how I create the npm releases.

It's not turtles all the way down, though. If you look way back, the 0.1ish releases of the compiler were written in CoffeeScript. I ported the compiler source over from CoffeeScript file by file as it matured and eventually got rid of the dependency entirely. Technically, you could start with the last release that was written in CoffeeScript and compile up to the current release.

[arghbleargh]

It appears to be very similar to icedcoffeescript: http://maxtaco.github.io/coffee-script/

[jashkenas]

Beautiful work. The `for parallel` notation is a really interesting way of writing it.

It seems like a number of folks in this thread have expressed interest in your process -- both of designing the feature set, and of gradually bootstrapping it away from CoffeeScript to become self-hosting. I'd love to hear more, if not here, then in a blog post, perhaps...

[rzimmerman]

Thanks! I'm a big admirer of your work, so it means a lot to me to hear that. I wrote up a little bit about this if you are still interested: http://blog.rzimmerman.me/how-kal-compiles-itself/