top | item 32323831

(no title)

zentrus | 3 years ago

Airflow helped my team out a lot a couple years ago mainly for the simplicity of the topdown UI-based view of a complicated ETL AND the ability to retry parts of the ETL.

We had lots of lessons learned. For instance, why does PythonOperator even exist? It takes a callable and thus you're likely not going to see good coding pattern emerge for something that needs to be 1000+ LoC. Instead, we just subclassed BaseOperator and used tried-and-true OO principles.

discuss

order

No comments yet.