top | item 36764651

(no title)

swordsmith8 | 2 years ago

Thanks for sharing! Seems like this is a dbt-centric lineage tool that surfaces failed tests in the lineage itself?

Unlike a data observability platform like Monte Carlo which proactively monitors data, am I correct in assuming that your solution is less focused on data observability (i.e. monitoring production data and conducting root cause analysis / impact analysis) and more on ensuring reliable CI/CD?

discuss

order

ersatz_username|2 years ago

I wouldn't personally draw such a bright line between monitoring and reliable CI/CD. That division definitely exists but partly as a product of the complexity introduced by fragmented data systems. In some ways an ideal world is one where the need for extraordinarily complex monitoring tools is actually pretty limited because we had tools to validate end to end data pipelines before making code changes if that makes sense.

We actually already do data monitoring as well although we haven't built the specific alerting features of Monte Carlo. There are quite a few tools that do that really well so it's not our focus at the moment.