I've been thinking why exactly async-await was chosen at the function level, and not the caller level.
I mean for languages that have event loops at their core, why isn't every function `async` by default? Let the caller decide how it wants to use the function.
`async` functions don't wait for a result of a `Future` unless `await` is used.
Instead of putting all that effort into putting `await` everywhere, why not invert that logic and make them `await` by default, and instead have the async-await syntax used by callers?
Like if one to were to write a transpiler to do this in dart, it might compile the following -
void main() {
Future<int> xFuture = async doSomething()
int x = doSomething()
assert(await xFuture == x);
}
int doSomething {
...
}
This might be stupid, but my solution to the "what color is your function problem" is to make every function async.
I looks an awful lot like what go does with its `go` syntax, but because you have an event loop with Future and Async Streams, you don't need to worry about dealing with CSP.
* Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought
* Async computing is harder to learn and confusing for those just picking up the language. There are some pretty major ergonomics issues that you have to solve if you want anything more than a DSL to achieve noteworthy adoption levels.
I have thinking something similar but with generators:
//A async candidate
fun read_lines(file):
for line in file:
yield line }
let lines = read_lines("hello.txt").await //turn async
I wonder why something like this is not used. Exist a bad interaction that could arise from this? Maybe about how nest stuff?
fun uppercase_lines(file):
for line in read_lines(file):
yield line
let lines = uppercase_lines("hello.txt").await //turn async, but also recursivelly read_lines?
I do Javascript and wish throwing quick scripts was a bit easier, which are made a bit harder with promises/async since now you have to separate your scripts and functions depending on their color. So I made a library to help me[1]:
const name = await swear(fetch('/some.json')).json().user.name;
console.log(name); // Francisco
const error = await swear(readFile('./error.log')).split('\n').pop();
console.log(error); // *latest error log message*
It makes all functions to look like blue functions, but internally they are all red. I made this by using `Proxy()`[2], then queuing the operations and waiting for any unfinished one on the last operation, which is always a `.then()` (since there's an `await`). It is fully compatible with native promises.
While I do not use it directly since adding a library to make syntax slightly shorter defeats the point, I've included it into some of my async libraries:
I love this about Go. All functions are simple and synchronous, but if you want to call them concurrently, just "go SlowThing()" and coordinate with a channel. Compare that to the async stuff in C#, Python, etc -- it's bolted on later, and you see double of everything.
None of this is really problematic at the top level view of things. But when you need to compose libraries or applications that make use of these things - yes, even channels in Go - you can start running into problems.
Especially if you don't actually control the process you're running in. This is why promises and async/await really exist. E.g. if you have code you need to fork off the main UI thread so you don't block it, but need to re-capture the UI thread so you can call UI update code when you're done with your long running task. async/await is so much cleaner here.
Async/await allow for much more complex control of flow than simple goroutines and channels. Sometimes it's necessary. Sometimes it isn't. It was necessary in nodejs. It's extremely useful in applications where you have a "main thread" that drives your application that you don't want to unnecessarily block. Go doesn't solve this problem out-the-box. There's a reason why c# adopted async/await despite having futures and all sorts of other tools.
But... don't bother too much trying to explain this to ppl who don't intuitively grok it right away... some just seem to never get it no matter how hard you try and explain it to them, it's like their brains are "wired differently" when it comes to reading and understanding code, they don't get the advantage and meaning of unification / universality / "one solution for many problems" etc.
Go is NOT my favorite programming language, but there's a stroke of simple genius in it that probably only got materialized bc its creators were left alone to work on it their way inside Google and just implemented their solution to things without being bothered by "language experts"...
EDIT+: not saying "colorless + channels" is always better or anything like that, all's tradeoffs... probably async/await code is much more readable than channels hell for many/most cases (but because it's less powerful - no true parallelism possible).
You can actually do the same thing as `go SlowThing() ` in C#, though it has more boilerplate. For example, you can do `Task.Factory.StartNew(() => SlowThing())`.
The thing is, Go doesn't have async functions, because it doesn't have await. That is why you don't have colored functions in Go: it doesn't support anything as advanced. Sure, the runtime does really cool things under the hood, but the Go programming model is more like Threads than async/await, because goroutines can't return data and they can't throw exceptions.
Go routines just act like threads. Something you can spawn in any language with the same guarantees. Even C/C++ have channel libraries. Channels just easily introduce complexity and channel-hell which is why they aren't all that popular as abstractions. Even in Go.
Frankly, I have a hard time imagining too many seasoned developers clutching channels to their chest as something superior to async/await. Async code with channels is not simple, you have in-channels, out-channels, and channels-over-channels. It's a pretty niche instrument working best when you only have uni-directional communication in your application.
For example, I've worked on channel code where I'd rather be debugging mutex deadlocks.
In fact there are not function at all, just like generators are not.
Also coroutines can be called from functions and vice versa. You don't have to choose.
The problem is not the color. The problem is the default paradigm is to be blocking I/O, and you add non blocking on top. It's way easier to make everything non blocking, then add blocking things on top.
It's not a matter of syntax, but of core paradigm.
Go and erlang were designed from the non blocking perspective from the get go.
But even go add to use something to manage async control flow, so it uses channels, which you don't need if you have coroutines.
Erland went deeper and made everything immutable, and baked in actors in the runtime. To me that's more impressive.
That's the benefit of integrating async in the runtime itself. It abstracts it from the language, which doesn't have to know about it. Just like a GC abstract memory handling.
I think this is just a matter of perspective and actually having different colors is a good thing:
Thinking in terms of functional programming / category theory, this coloring seems to boil down to working with different arrows. The coloring in this case would correspont to signifying the target category of the arrows `arr/lift` function (For example async functions in JS should be morphisms in smth like the Kleisli category of the `Promise` functor).
The issue now seems to be that there are no nice native language constructs in most languages to work with arrows. What's missing are arrow comprehensions - the bridge between the compositional notation used in functional languages and the more traditional way of writing things like `const x = ...`. The vanilla functions without `async/await` are arrow comprehensions for the identity monad.
The reason why the `async/await` syntax works so well as compared to using `.then` is that it is basically arrow comprehension for the promise monad.
"I think this is just a matter of perspective and actually having different colors is a good thing:"
I don't think it's a "good thing". I think it's a "cost". Even in nice functional languages, it's a cost, in that even with the nicest syntax there is you still get color cascades when you realizes deep in the middle of some deep function that you're in the wrong color.
There's a great case to be made that said nice functional languages, through a combination of ameliorating the costs and successfully obtaining benefits from the separation, mean that it's a good tradeoff.
Conventional imperative languages do not successfully obtain enough benefits from the specific color of "asynchronousness" (functional languages have many more colors than just that) to make it worth while. Covering it over with syntax wouldn't be enough, because it's not just a syntax problem. What you really need is to not have this particular color in the first place. (While arguably lacking other colors that you really need, but that's a story for a different day.)
Unless you refuse to pay even a little bit to avoid that color problem (which is a valid choice in some circumstances but I would submit a bad default choice for most code and most coders), we can increasingly see that there are languages that do not have the async color issue at all, and pay only modest performance prices to do so. As languages like Go and the Beam VM continue to progress, and are joined by other up-and-coming languages, even the problems with being unable to interact with C libraries fade as you can find something that does what you need for the most part. (And besides... personally I "blame" C rather than the languages trying to do better than C and then having trouble interacting with such primitive code.)
If you really need to run your computations in a separate context/arrow, making them explicit in the types is nice. Having general syntactic rules to build, compose and run them: great.
However, if you can avoid the additional arrow entirely: even better!
I mean Haskell of all languages has green threads and doesn’t need the entire promise shenanigans.
Even in Haskell I sometimes I build very specific DSLs for my domain to be interpreted in a separate step so I can freely fix my pure and impure code.
I have some vague knowledge of category theory, but not enough to follow your argument here. Especially the concept of an 'Arrow' could you explain more clearly what an arrow, and an arrow comprehension is?
This is something I ran into recently when learning about promises and async functions in javsscript. It took me a while to grok it, and I'm still not sure if I fully understand it, but we settled on using try/catch blocks with typescript's await keyword on calls we need to block on.
I wanted to use @usefultools/monads to have Maybe types, etc, but it's easier to just stick with promises throughout the chain. Essentially working with functions that return promises (like database calls with Mongoose) makes you convert your functions to also use promises. As far as I can tell there's no way to unwrap a promise (similar to Rust) without the await keyword inside a try block.
I don't know if this is exactly the best approach, but it's significantly less verbose than .then().catch() callbacks. I also learned about Promise.all() and Promise.allSettled() recently here [0].
I wish I could stick to futures with Rust or goroutines with golang. Bleh.
Classic article. In rxjs mapping over a sync array or async stream is exactly the same, for me the problem in the article is not such a big problem in practice for day to day work, especially with tools like rxjs which makes composing mixed code easy
I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge).
In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures.
What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer.
I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them).
I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.)
In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity).
I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.
I love the idea of coloring functions based on how "heavy" they are on the CPU or how long they take to execute on an "average client desktop/mobile browser".
This article made me angry then I first read it, and now it makes my blood boil. First off there where no colors in JavaScript pre ES6, there where only functions. You can call them first class functions, but those are still functions. A callback is just a function! Then we have side effects, but executing the side effect while returning a promise like in ES6 you are still executing the side effect, even if yo do not call .then() so nothing really changes.
So first you think there is a problem, but there are not any, but by solving the imagined problem you do actually create it.
You now have generator functions and "async" (with the async keyword) functions that always returns a promise, and normal functions.
I think you didn’t really understand the argument. The author is not saying that the language lacks first-class functions, in fact he requires it for his real point, which is that async and sync functions are not really compatible.
devxpy|5 years ago
I mean for languages that have event loops at their core, why isn't every function `async` by default? Let the caller decide how it wants to use the function.
`async` functions don't wait for a result of a `Future` unless `await` is used.
Instead of putting all that effort into putting `await` everywhere, why not invert that logic and make them `await` by default, and instead have the async-await syntax used by callers?
Like if one to were to write a transpiler to do this in dart, it might compile the following -
into - This might be stupid, but my solution to the "what color is your function problem" is to make every function async.I looks an awful lot like what go does with its `go` syntax, but because you have an event loop with Future and Async Streams, you don't need to worry about dealing with CSP.
chaorace|5 years ago
* Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought
* Async computing is harder to learn and confusing for those just picking up the language. There are some pretty major ergonomics issues that you have to solve if you want anything more than a DSL to achieve noteworthy adoption levels.
mamcx|5 years ago
franciscop|5 years ago
While I do not use it directly since adding a library to make syntax slightly shorter defeats the point, I've included it into some of my async libraries:
• File handler `files`: https://www.npmjs.com/package/files
• Simple command runner `atocha`: https://www.npmjs.com/package/atocha
• Enhanced fetch() `fch`: https://www.npmjs.com/package/fch
[1] https://www.npmjs.com/package/swear
[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
benhoyt|5 years ago
bcrosby95|5 years ago
None of this is really problematic at the top level view of things. But when you need to compose libraries or applications that make use of these things - yes, even channels in Go - you can start running into problems.
Especially if you don't actually control the process you're running in. This is why promises and async/await really exist. E.g. if you have code you need to fork off the main UI thread so you don't block it, but need to re-capture the UI thread so you can call UI update code when you're done with your long running task. async/await is so much cleaner here.
Async/await allow for much more complex control of flow than simple goroutines and channels. Sometimes it's necessary. Sometimes it isn't. It was necessary in nodejs. It's extremely useful in applications where you have a "main thread" that drives your application that you don't want to unnecessarily block. Go doesn't solve this problem out-the-box. There's a reason why c# adopted async/await despite having futures and all sorts of other tools.
aaai|5 years ago
But... don't bother too much trying to explain this to ppl who don't intuitively grok it right away... some just seem to never get it no matter how hard you try and explain it to them, it's like their brains are "wired differently" when it comes to reading and understanding code, they don't get the advantage and meaning of unification / universality / "one solution for many problems" etc.
Go is NOT my favorite programming language, but there's a stroke of simple genius in it that probably only got materialized bc its creators were left alone to work on it their way inside Google and just implemented their solution to things without being bothered by "language experts"...
EDIT+: not saying "colorless + channels" is always better or anything like that, all's tradeoffs... probably async/await code is much more readable than channels hell for many/most cases (but because it's less powerful - no true parallelism possible).
tsimionescu|5 years ago
The thing is, Go doesn't have async functions, because it doesn't have await. That is why you don't have colored functions in Go: it doesn't support anything as advanced. Sure, the runtime does really cool things under the hood, but the Go programming model is more like Threads than async/await, because goroutines can't return data and they can't throw exceptions.
hombre_fatal|5 years ago
Frankly, I have a hard time imagining too many seasoned developers clutching channels to their chest as something superior to async/await. Async code with channels is not simple, you have in-channels, out-channels, and channels-over-channels. It's a pretty niche instrument working best when you only have uni-directional communication in your application.
For example, I've worked on channel code where I'd rather be debugging mutex deadlocks.
ghostwriter|5 years ago
http://sdiehl.github.io/gevent-tutorial/
BiteCode_dev|5 years ago
In fact there are not function at all, just like generators are not.
Also coroutines can be called from functions and vice versa. You don't have to choose.
The problem is not the color. The problem is the default paradigm is to be blocking I/O, and you add non blocking on top. It's way easier to make everything non blocking, then add blocking things on top.
It's not a matter of syntax, but of core paradigm.
Go and erlang were designed from the non blocking perspective from the get go.
But even go add to use something to manage async control flow, so it uses channels, which you don't need if you have coroutines.
Erland went deeper and made everything immutable, and baked in actors in the runtime. To me that's more impressive.
bvrmn|5 years ago
https://www.youtube.com/watch?v=zeLToGnjIUM
dang|5 years ago
2018: https://news.ycombinator.com/item?id=16732948
discussed at the time: https://news.ycombinator.com/item?id=8984648
a bit more from the same day: https://news.ycombinator.com/item?id=8982494
smabie|5 years ago
afc|5 years ago
knrz|5 years ago
BiteCode_dev|5 years ago
Garlef|5 years ago
Thinking in terms of functional programming / category theory, this coloring seems to boil down to working with different arrows. The coloring in this case would correspont to signifying the target category of the arrows `arr/lift` function (For example async functions in JS should be morphisms in smth like the Kleisli category of the `Promise` functor).
The issue now seems to be that there are no nice native language constructs in most languages to work with arrows. What's missing are arrow comprehensions - the bridge between the compositional notation used in functional languages and the more traditional way of writing things like `const x = ...`. The vanilla functions without `async/await` are arrow comprehensions for the identity monad.
The reason why the `async/await` syntax works so well as compared to using `.then` is that it is basically arrow comprehension for the promise monad.
jerf|5 years ago
I don't think it's a "good thing". I think it's a "cost". Even in nice functional languages, it's a cost, in that even with the nicest syntax there is you still get color cascades when you realizes deep in the middle of some deep function that you're in the wrong color.
There's a great case to be made that said nice functional languages, through a combination of ameliorating the costs and successfully obtaining benefits from the separation, mean that it's a good tradeoff.
Conventional imperative languages do not successfully obtain enough benefits from the specific color of "asynchronousness" (functional languages have many more colors than just that) to make it worth while. Covering it over with syntax wouldn't be enough, because it's not just a syntax problem. What you really need is to not have this particular color in the first place. (While arguably lacking other colors that you really need, but that's a story for a different day.)
Unless you refuse to pay even a little bit to avoid that color problem (which is a valid choice in some circumstances but I would submit a bad default choice for most code and most coders), we can increasingly see that there are languages that do not have the async color issue at all, and pay only modest performance prices to do so. As languages like Go and the Beam VM continue to progress, and are joined by other up-and-coming languages, even the problems with being unable to interact with C libraries fade as you can find something that does what you need for the most part. (And besides... personally I "blame" C rather than the languages trying to do better than C and then having trouble interacting with such primitive code.)
sfvisser|5 years ago
However, if you can avoid the additional arrow entirely: even better!
I mean Haskell of all languages has green threads and doesn’t need the entire promise shenanigans.
Even in Haskell I sometimes I build very specific DSLs for my domain to be interpreted in a separate step so I can freely fix my pure and impure code.
rocqua|5 years ago
andrewzah|5 years ago
I wanted to use @usefultools/monads to have Maybe types, etc, but it's easier to just stick with promises throughout the chain. Essentially working with functions that return promises (like database calls with Mongoose) makes you convert your functions to also use promises. As far as I can tell there's no way to unwrap a promise (similar to Rust) without the await keyword inside a try block.
I don't know if this is exactly the best approach, but it's significantly less verbose than .then().catch() callbacks. I also learned about Promise.all() and Promise.allSettled() recently here [0].
I wish I could stick to futures with Rust or goroutines with golang. Bleh.
[0]: https://news.ycombinator.com/item?id=23223881
gorgoiler|5 years ago
His write up on the process is also interesting:
http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web...
KingOfCoders|5 years ago
hombre_fatal|5 years ago
For example, something as simple as:
joshribakoff|5 years ago
elwell|5 years ago
k__|5 years ago
afc|5 years ago
I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge).
In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures.
What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer.
I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them).
I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.)
Here is one example of where it is used: https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db...
In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity).
I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.
sergiotapia|5 years ago
waynesonfire|5 years ago
longtermd|5 years ago
saagarjha|5 years ago
z3t4|5 years ago
So first you think there is a problem, but there are not any, but by solving the imagined problem you do actually create it. You now have generator functions and "async" (with the async keyword) functions that always returns a promise, and normal functions.
saagarjha|5 years ago