Hey HN, Arch CEO here! Our team has been working at the intersection of data engineering and software engineering for a few years now with Meltano (https://meltano.com), and this year, the rise in Generative AI has made it clear that the bottleneck in unlocking the potential value of data has shifted from data integration on data teams to data engineering on software teams, so we’ve decided to do something about it.
What have you seen in the past five years that's made progress in this area? There's a lot of the "Modern Data Stack" tools that have certainly helped, but those generally feel separate from ML/AI workloads (usually).
DouweM|2 years ago
This post is about the high-level motivations and long-term vision for https://arch.dev; please let me know if any of it resonates or if you think I’m totally off the mark :) We’ve also got a post that goes into more detail on the Arch product itself: https://www.arch.dev/blog/announcing-arch-the-data-backend-f...
PaulHoule|2 years ago
tayloramurphy|2 years ago