[chrisjc]

Any thought about a pass-through server for various non-local databases?

It should be extremely simple for databases that support ADBC (for example Snowflake, PostgreSQL).

For others it might just be a matter of mapping DDL, DML, DQL, etc to a supported database protocol driver (JDBC, ODBC, etc). Of course this is where things may get challenging as it would become the responsibility of your server to convert result to Arrows (tables/streams/etc). But could potentially be delegated to "worker" Flight servers (not a Flight SQL server) and then the server could return/forward their Arrow results (Flight results).

Of course some of this is to some degree already possible through DuckDB's MySQL/Postgres Extensions.

I imagine this could also be useful for developing/testing locally?

It might also provide a way to interchange databases while potentially easing database migrations (vendor to vendor) if ADBC isn't supported by the vendor.

Another potential value-addition could be to provide SQL dialect management by providing Substrait conversions (or sqlglot but looks like the server is Java, so unsure if possible, maybe Graal?).

[philbe77]

Hi chrisjc, these are interesting thoughts. I was mainly interested in providing a server interface for DuckDB b/c of the performance I observed when running large OLAP workloads, and especially the cost savings vs cloud SaaS offerings. I think the big benefit here is that you can run the same workload at roughly equivalent performance - but for 80% less - from my tests...

I like your ideas, though. The server is written in C++.

I hope this helps...

Thanks!