[yevpats]

Firstly, congrats :) (Generalized) ingestion is a very hard problem because any abstraction that you come up with will always some limitations where you might need to fallback to writing code and have full access to the 3rd party APIs. But definitely in some cases generalized ingestion is much better then re-writing the same ingestion piece especially for complex connectors. Take a look at CloudQuery (https://github.com/cloudquery/cloudquery) open source high performance ELT framework powered by Apache Arrow (so you can write plugins in any language). (Maintainer here)

[karakanb]

couldn't agree more! I see ingestr more as a common-scenario solution rather than a general solution that solves all cases, kinda like how I treat shell oneliners instead of writing an applicataion in another language. I guess there's space for both approaches.

I'll definitely take a look at CloudQuery, thanks a lot for sharing!