top | item 23349755

(no title)

Peteris | 5 years ago

Airflow is an incredibly powerful framework to use in production, but a little unweildy for anything else.

You can use something like Kedro (https://github.com/quantumblacklabs/kedro) to get started building pipelines with pure Python functions. Kedro has its own pipeline visualiser and also has an Airflow plugin that can automatically help you generate airflow pipelines from Kedro pipelines.

https://github.com/quantumblacklabs/kedro-airflow

discuss

order

No comments yet.