And why restrict oneself to just two colours? Haskell monads also allow one to abstract over the "colour", such that one can write polymorphic code that works for any colour, which I think was the main objection from the original red/blue post.
Don't monads in themselves have the same problem of introducing "coloring" (where each monad is a different color) , so that functions which work for one monad won't work with easily compose with functions which work with another monad?
For example, isn't it true that you can't easily compose a function f :: a -> IO b with a function g :: b -> [c] (assuming you don't use unsafePerformIO, of course)?
Of course, there will be ways to do it (just as you can wrap a sync function in an async function, or .Wait() on an async function result to get a sync function), and Haskell's very high level of abstraction will make that wrapping easier than in a language like C#.
Fun Fact: The author of Koka (Daan Leijen) wrote a LaTeX-flavored Markdown editor called Madoko [1] using Koka! It is used to render the "Programming Z3" book [2], for instance.
Monads are not the same thing as effect types - the latter are most commonly modeled as Lawvere theories. They are at a different level of composability.
Regardless, this blogpost is about Rust which has yet to gain either feature, for well-known reasons (for one thing, monadic bind notation interacts badly w/ the multiple closure types in Rust). Talking about Haskell monads just does not seem all that relevant in this context.
The problem is that it is unclear that effect systems are something worth having in the first place. A type system could introduce many kinds of distinctions -- the unit allocates memory or not, the unit performs in a given worst-case time or not, etc. etc. -- but not everything it could do it also should. Every such syntactic distinction carries a cost with it (it tends to be viral), and for each specific distinction, we should first establish that the benefits gained outweigh those costs. Otherwise, it's just another property among many that a programming language may have, whose value is questionable.
Basically, in JS you could have multiple colors (with generators that imitate `do` notation pretty well) and you can obviously implement any monadic structure with bind you'd want.
The thing is: having less generic abstractions (and fewer colors) makes certain things (like debugging tooling and assumptions when reading code) much much nicer.
I'm not sure it's correct to understand monads as a kind of function coloring mechanism. Two reasons. First, monads aren't functions.
Second, the whole function coloring thing wasn't a complaint about limits on function composition. If it were, we might have to raise a complaint that arity is a kind of color. It was a complaint that functions of one color can't be called from inside functions of another color. And you don't really have an equivalent problem with using one monad inside the implementation of another monad.
Honestly, this sounds like somebody struck Achilles' heel of Rust and it hurts. They're neatly covering 3 stages of grief over these 3 articles, and I wouldn't be surprised if we could uncover the other 2 in the blogosphere somewhere.
> They're neatly covering 3 stages of grief over these 3 articles
No. It could only be something like three stages of grief if a single person were cycling among these positions. When it's just three different people writing three different blog posts, it's either three completely unrelated things, or a debate.
The two articles you list actually agree with each other - the second one is basically a more thorough argumentation of the points raised in the first one.
For example, the first article agrees that "async is colored"(despite the title), but that the big issue of "colored" functions, "You can only call a red function from within another red function", doesn't exist in Rust. This is also a major point in the second article.
I think a more accurate description would be that Rust async is informed by painful async implementations(Javascript) and has tried to avoid most of their shortcomings.
I actually have found all the angst and consternation around the "coloring" of Rust's async approach to be rather funny, because there are heaps of concepts in Rust which introduce something analogous to function coloring.
A good example would be lifetimes. Once you introduce lifetime parameters into a data type, it starts to introduce ballooning complexity into everything that touches it.
This is because the original article talked about a couple different things under the same title of a "coloring" problem:
1. JS has no way to wait for results of async functions in a sync context. This creates hard split between sync and async "colors". This is what most people agree is the major problem.
2. C# can avoid the split, but the syntax for using sync and async is different. The original article wasn't very clear about why this is bad, other than it exists.
The second case gets interpreted as either:
2a. Programming languages should be able to generically abstract over sync/async syntax.
2b. Programming languages shouldn't have a different sync/async syntax in the first place.
Rust's case is consistently:
- It generally doesn't have the problem #1. Sync code can wait for async. Async code can spawn sync on a threadpool. There are some gotchas and boilerplate, but this kind of "color" isn't an unsolvable dead-end like it is in JS.
- It has distinct syntax and execution model for async, which still counts as "color" (#2) per the original article. However, Rust can abstract over sync and async, so it doesn't have the problem #2a. The syntactical distinction is very intentional, so it doesn't consider #2b to be a problem at all.
These are three different opinions from three separate people. Please consider that the Rust community does not share one consciousness, as your post doesn't add anything useful to the conversation.
But they are a good thing, from a type perspective, they encode the idea that a side-effect is going to happen at the type level, and then force you as the caller to handle that fact or pass the buck to your caller.
Effect types are a signal for all programmers that "Here be dragons". They outline parts of your code which may not be safe to reorder during refactoring. And they can cover so much more than just asynchronous I/O. [1]
This article was a whole lot of "this pain you feel, cherish it, it's good".
It embraces a defect introduced for BC reasons as if it's sound engineering. It really isn't.
If you start over, there's no reason any blocking API should really BLOCK your program. Your functions can look completely synchronous, but yield to the event loop when they invoke a blocking API.
Basically like Erlang works.
Eventually it will be the case, but these kind of "love the flaw" attitudes are slowing us down.
In Erlang/Elixir, every function from another module is async by design, and the await is implicit.
This is by design to allow for hot code reloading. This is a very sound design choice because true concurrency is achieved through the creation of other processes (agents, genservers, tasks, ...).
You have nice things like Dynamic Supervisors to monitor those processes. And it works very well.
Try to do the same with uncancellable promises in Javascript/Typescript, or with channels in Go, while it's still possible, you'll find your code growing in complexity way quicker.
The actor model embedded in the language, like Erlang/Elixir, brings so much to the table, it's unfortunate that the author dismiss it without a second thought.
Actually the entire Erlang runtime is "async"(in the terms of the other mentioned languages) almost all the time since it's based on green threads/processes(stuff like a NIF might block of course), the module call is more of a lookup-current-latest-definition call to enable a well defined point for the hot-loading whereas local functions would more likely to be in the middle of a computation (and thus be more likely to be in a complicated-to-upgrade state).
async/await being implemented and popularized as it is today everywhere might just be our generations null-mistake since they're so error prone, they're great to show off parallel IO performance increases for classically synchronous languages but the programmer-burden of explicit await's (luckily IDE's/compilers can catch them) with "weird" side-effects was a big mistake.
The author of the article seems to be firmly in a specific category of developers so the dismissal seems quite in-character.
If you're not, here's the link [0]. It's a pretty influential post, it's introduced whole new terminology of "colored functions" into tech jargon that's starting to become reasonably commonly used. You should familiarize yourself with it if you haven't already.
The point the original post is making is that in languages that support asynchrony using callbacks, promises or async/await, they've split the language in two: the "synchronous" and "asynchronous" parts, and reconciling them is a pain. The only languages which avoids this split comfortably are Go and a few other more niche ones (notably BEAM languages like Erlang and Elixir).
This post makes the counter-argument that it's actually a good thing that functions are "colored" differently, as synchronous and asynchronous functions have different execution models, and that should be reflected in the language.
The association with "effect" and "async" is erroneous when discussing the topic at this level. Is a heap-read an effect? Is an L2 cache write an effect? Of course they are, but the machine abstracts these from us. We consider effects like `fetch(...)` an effect because of its associativity of remoteness and propensity to fail or change, but that's the incorrect lens. Instead, the lens should be purity, including purity of passed routines. If the network was rock solid (like mem reads) and we read a fixed document, suddenly that effect wouldn't feel like an effect. The color of the fn would go back to whichever is the pure form, and all would be well. We deem some effects effects because of programming constructs and performance, and other effects we don't even consider because we trust the underlying implementation of the machine. Consequently, I fundamentally don't fully align with the article's claim.
While I do generally agree that the color information is communicative, I think function names and input/output types are responsible for that type communication.
When OCaml effects land in 5.x, I look forward to using the effect keyword to achieve this goal. For better or worse, I plan to use it for much more as well--providing general implementations for _most things_, simply s.t. providing test doubles in integration tests becomes a piece of cake, with zero need for fancy test provisions (module mocking, DI, ...<insert usual suspects>)
In Zig allocators are always passed around explicitly and yet we have a completely opposite stance on function coloring [1].
In my opinion the argument doesn't really apply to async await when you get down to the details, but I need to think about it more to be able to articulate why I think so with precision.
It's not true that in zig allocators are always passed around explicitly. You can:
- declare a global allocator for your program and call it implicitly everywhere. This is very accepted by the community as bad practice for libraries but if you are writing end product who cares. In the end in most cases you will have to bootstrap your allocator against a global allocator anyways, and usually that in some form or another std.heap.page_allocator.
- bind an allocator into your struct when you build it. This makes the allocator "implicit" instead of explicit.
The important criteria about red/blue functions that a lot of people forget about is that one of the seams between the two colors is missing. As far as I can tell, for all of "red"/"blue" ish things in zig, (async, allocators, errors) there are clear and documented ways of annealing those seams in your code.
It's really interesting that Zig seems to have managed to avoid the "color" distinction for async. I'm curious to see how it works out in the end (and am a happy sponsor, to help that out!).
However, and maybe this is what you're alluding to with your mention of allocators, it seems that Zig has "colored" functions in its own way, with regard to allocation! If a certain function allocates, then to use it the caller will need to pass in an allocator, and that caller will need to have been given an allocator, etc. In other words, explicit allocation "infects" the codebase, too, right?
I can't help but see the parallels there. I think Zig's stance on explicit allocation is one of its strengths, but it does make me wonder if some compiler trickery can make it colorblind as well.
This is a good time to mention a Python experiment [1] that "covers up" Python 3.x's coloured asyncio with a synchronous wrapper that makes it more user-friendly. It's been posted here before [2]. I've pushed ahead with it professionally, and I think it's actually held up much better than I expected.
This idiom worked well in Python 2.x with Tornado. Python 3's batteries-included async unfortunately broke it partway (event loops are not reentrant [3], deliberately). However, in this Rust post's "turn-that-frown-upside-down" spirit, the Python 3.x port of tworoutines enforces a more deliberate coding style since it doesn't allow you to remain ignorant of asynchronous calls within an asynchronous context. This (arguably) mandates better code.
I don't claim Python is superior to Rust, or that hiding red/blue functions is a great idea everywhere. I did want to point out that dynamic languages allow some interesting alternatives to compiled languages. This is worth musing over here because of the author's stated bias for compiled languages (I think I agree), and his claim that red/blue unification would only be enabled by sophisticated compilers (not necessarily).
Rust pretty much had to add function coloring, because of its low-level nature.
Other languages can get away with it due to having a runtime, notably Go, and with the coming Project Loom, Java. For higher level languages, I do believe that coloring is not a useful concept, similarly to how you use a GC, instead of manual memory management. The runtime can “hijack” calls to IO or other blocking functions and can park a virtual thread, and resume execution afterwards automatically. There is no useful information in coloring in this case.
One point I massively miss from this discussion is retrofitting. Imagine the following situation: I have a webserver which calls a parser module which calls an external library which calls a closure in my code during parsing. Now this call needs to write to a log, which is async due to a file write or so.
To implement this I need to touch the webserver core, implement a breaking change in the parser library and get a new external library, just to get my log line in this function. This is a massive refactoring effort just to get a line in my log. It's purer and better style, yes. But now I need a few days for what should take five minutes.
If the codebase is designed from the ground-up with async in mind (which probably means to make basically every non-leaf async, to avoid the situation above), this is fine. If I'm working with an existing codebase, this does not work. I see why people don't want a fallback - it leads to bad code in cover. But allowing this and simply marking it via linter would make step-wise upgrading so much easier and alleviate 90% of the problems.
EDIT: My concerns are specific about JavaScript/TypeScript. I think Rust got this right.
> Now this call needs to write to a log, which is async due to a file write or so.
Well, think about, without having a specific language in mind. There are multiple ways you want everything to be executed:
1. The library should call your function and wait until both parsing and writing to the file is finished before executing any other action
2. The library should call your function and wait until both parsing and writing to the file is finished, but potentially execute other actions (e.g. other parsing for another request) in the meantime / in between.
3. The library should call your function and wait until parsing is finished but not wait until writing to the file is finished (which also means to disregard the filewrite result)
The only way to really control how things happen is to have the library be aware of potential concurrency and that is true for Rust as well. There is no other solution, except for "hacks" which, when used, will make it impossible to understand what a piece of code does, without reading _all_ the code. Similar to global mutable variables (because that is what it would be).
If you application is small or not so critical, then you might indeed get away with a hack, but I think the energy should be focussed on improving languages so that libraries can easily be rewritten to embrace concurrency, rather than allowing for more hacks.
> To implement this I need to touch the webserver core, implement a breaking change in the parser library and get a new external library, just to get my log line in this function. This is a massive refactoring effort just to get a line in my log.
Not quite. It's very easy in Rust to purposefully block on an async call in sync-only code, or spawn blocking code on a side thread in async code and let it "join" whenever the async code block gets executed. Of course you'll get less-than-optimal performance, but no program-wide refactoring is required.
One of the best retrofitting effort there is is Java's Loom IMHO.
Instead of trying to solve the IO bottleneck on a few threads with async, it just reduces the Thread footprint and allows you to have millions of them.
Good'ol Tomcat is going to massively improve its IO concurrency overnight with just a simple `Thread.builder().virtual().factory()`.
> webserver which calls a parser module which calls an external library
There's your problem right there. Nested calls are killing us.
That parser module should be a pure transformation function. If it has requirements, your main webserver should be fetching it. If you need to first query the parser to know if it has any requirements for this call, you can do so as a separate pure function. That query will then respond with a value that your webserver can then use to call the external library with. When the impure external library responds, with no external changes, you can now pass that into the parser module to get your parsed object out.
Now when you want to add logging to the external library, ideally you can do so at the webserver level, but otherwise you only need to add logging to the external library, and your parser doesn't change at all.
1. Languages with "coloured functions" usually don't inform you of effects (except for Haskell and languages like it). They allow expressing equivalent semantics with the other colour, which doesn't distinguish effects.
2. Expressing anything in the type system always has a cost. That it can express something doesn't mean that it should. The only way to know whether expressing this difference -- assuming you could (which, again, most languages with coloured functions don't do) -- is to empirically show the benefits outweigh the cost, not just hypothesise that they do, which just amounts to "I like it this way." "I like it" is perfectly fine, but it's not the same as "it's a good thing."
> Inconvenient knowledge is better than convenient ignorance
Can't disagree with this statement more. I've worked with a coloured language for a very long time (C#) and also recently worked a lot with Go and both have huge advantages and disadvantages. Go is not conveniently ignorant. The green thread model which Go uses is actually a very elegant solution where one can write simple code without leaking abstractions down the stack as it happens to be with red and blue functions. It's not ignorance. It's focus! Only the function which deals with async code has to know and think about it. Calling code doesn't need to adapt itself to the inner workings of another function which it has no control of. I quite like it and don't see any particular problem or traps with it. Quite the opposite, as a very experienced C# developer I see a lot of less experienced developers do very fatal mistakes with async/await as it's so easy to write code which can end up in a deadlock or exhaust resources and run into weird issues which only a person with a lot of experience would be able to spot and fix. I think Go's model is much more tolerant and forgiving to beginner mistakes which means that writing fatal code tends to be a lot harder and almost impossible if not done intentionally.
> I quite like it and don't see any particular problem or traps with it.
While Go's model is certainly a valid point in the design space and keeps things nice and simple, there are of course going to be many trade-offs. For example, low-level control is very much lost. There are no opportunities for custom schedulers, optimising how non-blocking calls are made, or easily dealing with blocking syscalls (or foreign libraries that call them). It would not be the right approach for Rust.
Don't know about go, but in elixir "We have to use Task.await", is like, you can use it if you want exactly those semantics, just because it has await and task in its name doesn't mean you have to - although simple in usage it's quite a higher level abstraction inside the BEAM - the base unit is the "process", and you can make any sort of asynchronous or synchronous flow on top of that.
But even with Tasks, they can be called inside other processes and then instead of having to wait for all tasks to finish they'll be delivered one by one to that process mail box and you can decide at all points of execution what to do. Of course if you're running your code like a sequential block... You'll have to wait until they sequentially finish?
While I agree with his points in principle, Rust isn't actually a great example to use.
Rust allows to simply spawn threads, right? So a function not being marked 'async' doesn't necessarily give any guarantees about potential side effects.
Spending most of my time writing Elm, having side-effects present in the type system is great. But colored functions in Rust (as well as most other async/await langs) only creates a devide between functions that provide a specific kind of asynchrony, and those which do not.
So async/await improves the syntax of writing a specific kind of async code, at the cost of composability (the red/blue divide). It does not, however, make any guarantees about the asynchronous nature of your code.
Interesting. This author took a different interpretation of the original red/blue functions article than I did. I understood the original to mean "the distinction between red and blue functions is a bad thing when not made visible in the type system". Perhaps that's my Haskell bias showing and I missed the original point.
Haskell has one of the _clearest_ distinctions between red/blue functions that I know. Functions that perform I/O (sans unsafePerformIO and other strange things) are very explicit about it (by returning an `IO` monad).
Most other languages (static or not) are equally explicit (returning a `Promise<A>` and not an `IO a` but it's very similar. Most differences are because call-by-need vs. call-by-value language semantics (Haskell being non-strict and the I/O not coming out of "thin air" but coming from somewhere - but that's kind of a different point than function color type)
The colors are important if you are going to do async. They tell you significant details about the functions you are using. The issues most people have with colored functions aren't that they are have colors. It's that the colors don't compose. The languages that deal the best with this pick a single color and commit to it. Go and any BEAM language make every function async effectively and handle the coordination for you. They succeed because their runtimes will do a better job than you will 99% of the time. For the rest they give you some tools to reason about it.
Rust however doesn't come with it's own runtime it's meant for the kind of development where you are building your own runtime or bringing one. For that reason the colors (i.e. types) are surfacing all of that complexity for you and helping you handle it.
The bigger problem as I see it is that the rush for async has left a lot of crates choosing either sync or async only and so finding the crate that for your particular use case can be hit or miss. Half the db libraries may be sync only and half of the http libraries may be async only and then you have to figure out how to marry the two. If you can live in one or the other world only you are fine. It's when you have to live in both simultaneously that things get hairy.
This author claims there shouldn't be red and blue functions at all, instead concurrent paths should be handled by a Nursery object, so that no background tasks can ever be dropped by the runtime, but always return to the nursery.
[+] [-] willtim|5 years ago|reply
Microsoft's Koka is an example of a language that further empraces effect types and makes them easier to use: https://koka-lang.github.io/koka/doc/index.html
[+] [-] simiones|5 years ago|reply
For example, isn't it true that you can't easily compose a function f :: a -> IO b with a function g :: b -> [c] (assuming you don't use unsafePerformIO, of course)?
Of course, there will be ways to do it (just as you can wrap a sync function in an async function, or .Wait() on an async function result to get a sync function), and Haskell's very high level of abstraction will make that wrapping easier than in a language like C#.
[+] [-] benrbray|5 years ago|reply
[1] http://madoko.org/reference.html
[2] https://theory.stanford.edu/~nikolaj/programmingz3.html
[+] [-] zozbot234|5 years ago|reply
Regardless, this blogpost is about Rust which has yet to gain either feature, for well-known reasons (for one thing, monadic bind notation interacts badly w/ the multiple closure types in Rust). Talking about Haskell monads just does not seem all that relevant in this context.
[+] [-] pron|5 years ago|reply
[+] [-] the_duke|5 years ago|reply
[1] https://github.com/unisonweb/unison
[+] [-] inglor|5 years ago|reply
The thing is: having less generic abstractions (and fewer colors) makes certain things (like debugging tooling and assumptions when reading code) much much nicer.
[+] [-] mumblemumble|5 years ago|reply
Second, the whole function coloring thing wasn't a complaint about limits on function composition. If it were, we might have to raise a complaint that arity is a kind of color. It was a complaint that functions of one color can't be called from inside functions of another color. And you don't really have an equivalent problem with using one monad inside the implementation of another monad.
[+] [-] diragon|5 years ago|reply
Then "Rust async is colored, and that’s not a big deal" https://morestina.net/blog/1686/rust-async-is-colored
And now it's a good thing.
Honestly, this sounds like somebody struck Achilles' heel of Rust and it hurts. They're neatly covering 3 stages of grief over these 3 articles, and I wouldn't be surprised if we could uncover the other 2 in the blogosphere somewhere.
[+] [-] mumblemumble|5 years ago|reply
No. It could only be something like three stages of grief if a single person were cycling among these positions. When it's just three different people writing three different blog posts, it's either three completely unrelated things, or a debate.
[+] [-] Sacho|5 years ago|reply
For example, the first article agrees that "async is colored"(despite the title), but that the big issue of "colored" functions, "You can only call a red function from within another red function", doesn't exist in Rust. This is also a major point in the second article.
I think a more accurate description would be that Rust async is informed by painful async implementations(Javascript) and has tried to avoid most of their shortcomings.
[+] [-] skohan|5 years ago|reply
A good example would be lifetimes. Once you introduce lifetime parameters into a data type, it starts to introduce ballooning complexity into everything that touches it.
[+] [-] pornel|5 years ago|reply
1. JS has no way to wait for results of async functions in a sync context. This creates hard split between sync and async "colors". This is what most people agree is the major problem.
2. C# can avoid the split, but the syntax for using sync and async is different. The original article wasn't very clear about why this is bad, other than it exists.
The second case gets interpreted as either:
2a. Programming languages should be able to generically abstract over sync/async syntax.
2b. Programming languages shouldn't have a different sync/async syntax in the first place.
Rust's case is consistently:
- It generally doesn't have the problem #1. Sync code can wait for async. Async code can spawn sync on a threadpool. There are some gotchas and boilerplate, but this kind of "color" isn't an unsolvable dead-end like it is in JS.
- It has distinct syntax and execution model for async, which still counts as "color" (#2) per the original article. However, Rust can abstract over sync and async, so it doesn't have the problem #2a. The syntactical distinction is very intentional, so it doesn't consider #2b to be a problem at all.
[+] [-] tadfisher|5 years ago|reply
[+] [-] jackcviers3|5 years ago|reply
Effect types are a signal for all programmers that "Here be dragons". They outline parts of your code which may not be safe to reorder during refactoring. And they can cover so much more than just asynchronous I/O. [1]
1. https://en.wikipedia.org/wiki/Effect_system
[+] [-] slver|5 years ago|reply
It embraces a defect introduced for BC reasons as if it's sound engineering. It really isn't.
If you start over, there's no reason any blocking API should really BLOCK your program. Your functions can look completely synchronous, but yield to the event loop when they invoke a blocking API.
Basically like Erlang works.
Eventually it will be the case, but these kind of "love the flaw" attitudes are slowing us down.
[+] [-] linkdd|5 years ago|reply
This is by design to allow for hot code reloading. This is a very sound design choice because true concurrency is achieved through the creation of other processes (agents, genservers, tasks, ...).
You have nice things like Dynamic Supervisors to monitor those processes. And it works very well.
Try to do the same with uncancellable promises in Javascript/Typescript, or with channels in Go, while it's still possible, you'll find your code growing in complexity way quicker.
The actor model embedded in the language, like Erlang/Elixir, brings so much to the table, it's unfortunate that the author dismiss it without a second thought.
[+] [-] whizzter|5 years ago|reply
async/await being implemented and popularized as it is today everywhere might just be our generations null-mistake since they're so error prone, they're great to show off parallel IO performance increases for classically synchronous languages but the programmer-burden of explicit await's (luckily IDE's/compilers can catch them) with "weird" side-effects was a big mistake.
The author of the article seems to be firmly in a specific category of developers so the dismissal seems quite in-character.
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] bcrosby95|5 years ago|reply
This statement confuses me. What if I never spawn a process? What if the process I spawn makes function calls into other purely functional modules?
[+] [-] quelltext|5 years ago|reply
We are?
[+] [-] OskarS|5 years ago|reply
The point the original post is making is that in languages that support asynchrony using callbacks, promises or async/await, they've split the language in two: the "synchronous" and "asynchronous" parts, and reconciling them is a pain. The only languages which avoids this split comfortably are Go and a few other more niche ones (notably BEAM languages like Erlang and Elixir).
This post makes the counter-argument that it's actually a good thing that functions are "colored" differently, as synchronous and asynchronous functions have different execution models, and that should be reflected in the language.
[0]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
[+] [-] ghengeveld|5 years ago|reply
[+] [-] brailsafe|5 years ago|reply
[+] [-] rollulus|5 years ago|reply
[+] [-] agumonkey|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] cdaringe|5 years ago|reply
The association with "effect" and "async" is erroneous when discussing the topic at this level. Is a heap-read an effect? Is an L2 cache write an effect? Of course they are, but the machine abstracts these from us. We consider effects like `fetch(...)` an effect because of its associativity of remoteness and propensity to fail or change, but that's the incorrect lens. Instead, the lens should be purity, including purity of passed routines. If the network was rock solid (like mem reads) and we read a fixed document, suddenly that effect wouldn't feel like an effect. The color of the fn would go back to whichever is the pure form, and all would be well. We deem some effects effects because of programming constructs and performance, and other effects we don't even consider because we trust the underlying implementation of the machine. Consequently, I fundamentally don't fully align with the article's claim.
While I do generally agree that the color information is communicative, I think function names and input/output types are responsible for that type communication.
When OCaml effects land in 5.x, I look forward to using the effect keyword to achieve this goal. For better or worse, I plan to use it for much more as well--providing general implementations for _most things_, simply s.t. providing test doubles in integration tests becomes a piece of cake, with zero need for fancy test provisions (module mocking, DI, ...<insert usual suspects>)
[+] [-] kristoff_it|5 years ago|reply
In my opinion the argument doesn't really apply to async await when you get down to the details, but I need to think about it more to be able to articulate why I think so with precision.
[1] https://kristoff.it/blog/zig-colorblind-async-await/
[+] [-] dnautics|5 years ago|reply
- declare a global allocator for your program and call it implicitly everywhere. This is very accepted by the community as bad practice for libraries but if you are writing end product who cares. In the end in most cases you will have to bootstrap your allocator against a global allocator anyways, and usually that in some form or another std.heap.page_allocator.
- bind an allocator into your struct when you build it. This makes the allocator "implicit" instead of explicit.
The important criteria about red/blue functions that a lot of people forget about is that one of the seams between the two colors is missing. As far as I can tell, for all of "red"/"blue" ish things in zig, (async, allocators, errors) there are clear and documented ways of annealing those seams in your code.
[+] [-] losvedir|5 years ago|reply
However, and maybe this is what you're alluding to with your mention of allocators, it seems that Zig has "colored" functions in its own way, with regard to allocation! If a certain function allocates, then to use it the caller will need to pass in an allocator, and that caller will need to have been given an allocator, etc. In other words, explicit allocation "infects" the codebase, too, right?
I can't help but see the parallels there. I think Zig's stance on explicit allocation is one of its strengths, but it does make me wonder if some compiler trickery can make it colorblind as well.
[+] [-] gsmecher|5 years ago|reply
This idiom worked well in Python 2.x with Tornado. Python 3's batteries-included async unfortunately broke it partway (event loops are not reentrant [3], deliberately). However, in this Rust post's "turn-that-frown-upside-down" spirit, the Python 3.x port of tworoutines enforces a more deliberate coding style since it doesn't allow you to remain ignorant of asynchronous calls within an asynchronous context. This (arguably) mandates better code.
I don't claim Python is superior to Rust, or that hiding red/blue functions is a great idea everywhere. I did want to point out that dynamic languages allow some interesting alternatives to compiled languages. This is worth musing over here because of the author's stated bias for compiled languages (I think I agree), and his claim that red/blue unification would only be enabled by sophisticated compilers (not necessarily).
[1]: http://threespeedlogic.com/python-tworoutines.html [2]: https://news.ycombinator.com/item?id=18516121 [3]: https://bugs.python.org/issue22239
[+] [-] kaba0|5 years ago|reply
[+] [-] faitswulff|5 years ago|reply
As a new Rust user myself, this kind of snobbery is still off-putting.
[+] [-] coldtea|5 years ago|reply
As if you matter? The conclusion could be stripped, and the article would still make its point.
[+] [-] Sebb767|5 years ago|reply
To implement this I need to touch the webserver core, implement a breaking change in the parser library and get a new external library, just to get my log line in this function. This is a massive refactoring effort just to get a line in my log. It's purer and better style, yes. But now I need a few days for what should take five minutes.
If the codebase is designed from the ground-up with async in mind (which probably means to make basically every non-leaf async, to avoid the situation above), this is fine. If I'm working with an existing codebase, this does not work. I see why people don't want a fallback - it leads to bad code in cover. But allowing this and simply marking it via linter would make step-wise upgrading so much easier and alleviate 90% of the problems.
EDIT: My concerns are specific about JavaScript/TypeScript. I think Rust got this right.
[+] [-] valenterry|5 years ago|reply
Well, think about, without having a specific language in mind. There are multiple ways you want everything to be executed:
1. The library should call your function and wait until both parsing and writing to the file is finished before executing any other action
2. The library should call your function and wait until both parsing and writing to the file is finished, but potentially execute other actions (e.g. other parsing for another request) in the meantime / in between.
3. The library should call your function and wait until parsing is finished but not wait until writing to the file is finished (which also means to disregard the filewrite result)
The only way to really control how things happen is to have the library be aware of potential concurrency and that is true for Rust as well. There is no other solution, except for "hacks" which, when used, will make it impossible to understand what a piece of code does, without reading _all_ the code. Similar to global mutable variables (because that is what it would be).
If you application is small or not so critical, then you might indeed get away with a hack, but I think the energy should be focussed on improving languages so that libraries can easily be rewritten to embrace concurrency, rather than allowing for more hacks.
[+] [-] zozbot234|5 years ago|reply
Not quite. It's very easy in Rust to purposefully block on an async call in sync-only code, or spawn blocking code on a side thread in async code and let it "join" whenever the async code block gets executed. Of course you'll get less-than-optimal performance, but no program-wide refactoring is required.
[+] [-] BenoitP|5 years ago|reply
Instead of trying to solve the IO bottleneck on a few threads with async, it just reduces the Thread footprint and allows you to have millions of them.
Good'ol Tomcat is going to massively improve its IO concurrency overnight with just a simple `Thread.builder().virtual().factory()`.
[+] [-] ItsMonkk|5 years ago|reply
There's your problem right there. Nested calls are killing us.
That parser module should be a pure transformation function. If it has requirements, your main webserver should be fetching it. If you need to first query the parser to know if it has any requirements for this call, you can do so as a separate pure function. That query will then respond with a value that your webserver can then use to call the external library with. When the impure external library responds, with no external changes, you can now pass that into the parser module to get your parsed object out.
Now when you want to add logging to the external library, ideally you can do so at the webserver level, but otherwise you only need to add logging to the external library, and your parser doesn't change at all.
[+] [-] kbd|5 years ago|reply
https://kristoff.it/blog/zig-colorblind-async-await/
I highly recommend watching Andrew Kelley's video (linked in that article) on this topic:
https://youtu.be/zeLToGnjIUM
[+] [-] pron|5 years ago|reply
1. Languages with "coloured functions" usually don't inform you of effects (except for Haskell and languages like it). They allow expressing equivalent semantics with the other colour, which doesn't distinguish effects.
2. Expressing anything in the type system always has a cost. That it can express something doesn't mean that it should. The only way to know whether expressing this difference -- assuming you could (which, again, most languages with coloured functions don't do) -- is to empirically show the benefits outweigh the cost, not just hypothesise that they do, which just amounts to "I like it this way." "I like it" is perfectly fine, but it's not the same as "it's a good thing."
[+] [-] dustinmoris|5 years ago|reply
Can't disagree with this statement more. I've worked with a coloured language for a very long time (C#) and also recently worked a lot with Go and both have huge advantages and disadvantages. Go is not conveniently ignorant. The green thread model which Go uses is actually a very elegant solution where one can write simple code without leaking abstractions down the stack as it happens to be with red and blue functions. It's not ignorance. It's focus! Only the function which deals with async code has to know and think about it. Calling code doesn't need to adapt itself to the inner workings of another function which it has no control of. I quite like it and don't see any particular problem or traps with it. Quite the opposite, as a very experienced C# developer I see a lot of less experienced developers do very fatal mistakes with async/await as it's so easy to write code which can end up in a deadlock or exhaust resources and run into weird issues which only a person with a lot of experience would be able to spot and fix. I think Go's model is much more tolerant and forgiving to beginner mistakes which means that writing fatal code tends to be a lot harder and almost impossible if not done intentionally.
[+] [-] willtim|5 years ago|reply
While Go's model is certainly a valid point in the design space and keeps things nice and simple, there are of course going to be many trade-offs. For example, low-level control is very much lost. There are no opportunities for custom schedulers, optimising how non-blocking calls are made, or easily dealing with blocking syscalls (or foreign libraries that call them). It would not be the right approach for Rust.
[+] [-] hnedeotes|5 years ago|reply
Don't know about go, but in elixir "We have to use Task.await", is like, you can use it if you want exactly those semantics, just because it has await and task in its name doesn't mean you have to - although simple in usage it's quite a higher level abstraction inside the BEAM - the base unit is the "process", and you can make any sort of asynchronous or synchronous flow on top of that.
But even with Tasks, they can be called inside other processes and then instead of having to wait for all tasks to finish they'll be delivered one by one to that process mail box and you can decide at all points of execution what to do. Of course if you're running your code like a sequential block... You'll have to wait until they sequentially finish?
[+] [-] Skinney|5 years ago|reply
Rust allows to simply spawn threads, right? So a function not being marked 'async' doesn't necessarily give any guarantees about potential side effects.
Spending most of my time writing Elm, having side-effects present in the type system is great. But colored functions in Rust (as well as most other async/await langs) only creates a devide between functions that provide a specific kind of asynchrony, and those which do not.
So async/await improves the syntax of writing a specific kind of async code, at the cost of composability (the red/blue divide). It does not, however, make any guarantees about the asynchronous nature of your code.
[+] [-] tome|5 years ago|reply
[+] [-] inglor|5 years ago|reply
Most other languages (static or not) are equally explicit (returning a `Promise<A>` and not an `IO a` but it's very similar. Most differences are because call-by-need vs. call-by-value language semantics (Haskell being non-strict and the I/O not coming out of "thin air" but coming from somewhere - but that's kind of a different point than function color type)
[+] [-] zaphar|5 years ago|reply
Rust however doesn't come with it's own runtime it's meant for the kind of development where you are building your own runtime or bringing one. For that reason the colors (i.e. types) are surfacing all of that complexity for you and helping you handle it.
The bigger problem as I see it is that the rush for async has left a lot of crates choosing either sync or async only and so finding the crate that for your particular use case can be hit or miss. Half the db libraries may be sync only and half of the http libraries may be async only and then you have to figure out how to marry the two. If you can live in one or the other world only you are fine. It's when you have to live in both simultaneously that things get hairy.
[+] [-] mlatu|5 years ago|reply
https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
This author claims there shouldn't be red and blue functions at all, instead concurrent paths should be handled by a Nursery object, so that no background tasks can ever be dropped by the runtime, but always return to the nursery.