[eugenhotaj]

This is neat for toy problems but I don't see it working well for "real" pipelines. The magical DAG creation is going to be super hard to wrap your head around and even worse to debug.

This reminds me of an internal Google tool for doing async programming in Java (ProducerGraph or something). The idea was that you'd just write annotated functions and the framework would handle all the async stuff. Wasted many thousands of engineering hours while giving an even worse experience than manipulating futures directly.

[elijahbenizzy]

One of the authors here — I think that systems that produce things magically often run into this burden. That doesn’t mean that the magic is bad, per se, rather that the thought put into operationalizing it (the user experience) is often << than that put into executing it (the engine).

In our case there’s very little magic, and a dead simple engine (so far). We (from a DS perspective) have found that expressing transforms in this manner simplifies the code making it well worth the added layer of abstraction. As a platform team we’ve focused on debugging (viz, etc…), but this is part of the reason we open sourced it! The more voices we get the more operational concerns we can iron out to avoid painful debugging experiences.

[timkpaine]

A lot of these libraries also suffer from really ugly syntax, vs e.g. my library which overloads operators and attempts to "transparently" plug in to python's native syntactic sugar https://github.com/timkpaine/tributary

[krawczstef]

Looks like an interesting (& powerful) library - what problem is it trying to solve exactly? That wasn't clear from the README (the greek's library doesn't shed any more light on things either).