top | item 14067053

(no title)

bigzen | 9 years ago

Airflow might be primarily intended for workflows, but scheduling is fairly robust. But you also say that you want more than just simple Cron so I suggest taking another look.

What I really appreciate about Airflow is that it is self documenting. Anyone can logon and see the exact workflow, schedule of that workflow, the status of last few job runs and the python code that ran if they desire. Airflow documentation itself is good and getting better.

It can run a simple cron job but it also gives you room to build it out into much more. We use our Airflow scheduler to call simple API endpoints and then that same server does daily loads into our data warehouse.

Also 1.8 just dropped to stable and it is a huge improvement upon earlier versions.

discuss

order

No comments yet.