Task.js: Beautiful Concurrency for JavaScript

[pyalot2]

In python we've been having a yield statement for a long time. By far and large, it has not solved the issue of cooperative multitasking. Using it only leads to near incomprehensive code where everything becomes an iterable, and every call becomes an iteration.

Co-routines solve cooperative multitasking. Which is why they're now being discussed for python in a fashion that doesn't rely on C-stack hacks.

[kevingadd]

A better way to think of this is in terms of degrees of improvement.

Going from pure callback passing to yield-based cooperative multitasking increases clarity and removes a lot of opportunities for mistakes, so in practice it ends up being a win - but just like callback passing, you can still hit a point where it's too complicated or you're having to do too much.

It's only natural that going from custom task schedulers built on 'yield' to language-native mechanisms for cooperative multitasking improves things further. Even that, though, will likely not 'solve the issue', just make it considerably easier to tackle for individual applications. There's no One True Scheduler or One True Task Primitive, at least as far as I know: everyone seems to want something slightly different.

[themckman]

I recently just had great success using coroutines to build a sort of function pipeline that can work in both code and at the command line with minimal effort; I'm doing a lot of text processing. A contrived example that prints the numbers in range(0, 100) ends up looking something like this:

  @producer
  def prod():
    for i in range(0,100):
      yield i

  @producer
  @consumer
  def middleman():
    for i in (yield):
      yield i

  @consumer
  def cons():
    for i in (yield):
      print i

Producers can yield values and consumers can use yield to get their input. I will fully admit that code, abstracted away by those decorators, is not the prettiest and took a minute to understand when writing. The biggest gotcha was with both a producer and consumer. You have to be very careful to capture the return value of the generator.send(input) call as it ends up being the item of the first yield. Also, pure consumers will immediately throw a StopIteration exception after sending the input. I'm very glad, however, that I stumbled upon using generators in this way as it results in a very clean implementation at this level. The internals are a little dicey, however, which I think is what you're eluding to.

I use the functions in multiprocessing.Process objects as the target by first binding them to pipes and also without binding them to anything, in which case they use stdin/stdout.

[lucian1900]

I would say it has almost entirely solved the problem. Twisted with inlineCallbacks makes writing multitasking programs very clear and obvious. Python 3.3 makes it even better by allowing `return` and requiring less wrapping with `yield from`.

Implicit stack coroutines (like greenlets) can always be built on top if necessary, but they tend to require horrible things like monkey patching to work together with existing synchronous, blocking code.

[derefr]

Huh; I wonder if a slight tweak to this could be used to make yield act like Erlang's receive operator? Presuming the library assigns every task a handle (a PID), that tasks could .send() messages to other tasks, etc., .receive() could be implemented by the library just expecting you to yield a set of patterns, then blocking your task until the library has a message to route to it that matches one of those patterns.

[fzzzy]

Yep. I wrote some code a few years ago that does this. I wrote it before I knew about task.js and should rewrite it using task.js, removing my handrolled clone of it.

https://github.com/fzzzy/pavel.js

There is also another slightly different scheduler that I started that includes a multithreaded scheduler with webworkers and works in a browser. This version, because it is designed to work in stock browsers, has a callback api and does not use generators, but task.js could easily be layered on top.

https://github.com/fzzzy/actors.js

Please excuse the use of an explicitly provided "pattern" string in both of these. I did it for simplicity. It would be possible to port my concept of shapes from my python-actors project to js, but the runtime overhead of using explicit strings is extremely low and it makes the code very simple.

[kevingadd]

I think in practice you could just do this with encapsulation - i.e. you'd create channel objects and yield on them, instead of assigning handles directly to tasks.

[mrspeaker]

I wish Chrome would start implementing some of the ES6 stuff (especially arrow function syntax...mmmm): does anyone know where we can see that kinda status?

[fzzzy]

You can see some es6 features that are already in v8 and accessible from node here:

http://stackoverflow.com/questions/13351965/what-does-node-h...

No generators yet but they are currently being worked on.

[just2n]

Check the Chromium blog and their bug tracker.

If this were implemented, it would be hidden behind the experimental JS flag anyway.

[ger_phpmagazin]

This is a year old. Now what?