[troiskaer]

How does Kedro compare to MLFlow and Metaflow?

[joelschw]

Kedro sort of fits into a niche where it just overlaps somewhat with 'orchestrators' like Prefect, Metaflow, Dagster, Airflow and others. What makes it slightly different is that it it is focused on the rapid development journey to production, providing guardrails for teams to co-develop ML projects in a way that nudges software engineering best practice and clean code.

The 'finished article' in many cases should be deployed in production in one of those tools which provide specialised bells and whistles like scheduling, monitoring and observability.

Regarding MLFlow, there is also a slight overlap in terms of experimentation, but not things like model serving. Kedro has a mechanism to track experiments, but it's more designed to give users with zero infrastructure something for free out of the box. It's been built in a way that it can be repurposed for more dedicated experiment tracking tools - the folks at neptune.ai built their own plug-in for this purpose: https://docs.neptune.ai/integrations-and-supported-tools/aut...

[troiskaer]

Seems like Kedro has a similar thesis to Metaflow - I will look into it.

[idomi]

I believe essentially all of the tools mentioned here are focusing on the engineering persona and not the Data Scientist. Writing classes and functions isn't the jargon of a Data Scientist. At Ploomber we tried to put the Data Scientists in the center of everything, helping them to work together with OPS. Check it out! https://github.com/ploomber/ploomber

[troiskaer]

thanks for the unnecessary advertisement.

[stevesimmons]

And Prefect