More integrations are great. Anyway, the "this is awesome" moment (for me) will be when you could mix row- and column-oriented tables in Postgres, a bit like Timescale but native Postgres and well done. Hopefully one day.
I want MPP HTAP where SQL inserts/COPYs store data in three(!) formats:
- row-based (low latency insert, fast row-based indexed query for single-row OLTP)
- columnar-based (slow inserts/updates, fast aggregates/projections)
- iceberg-columnar-based (better OLAP price/performance and less lockin than native columnar)
And for SELECTs the query engine picks which storage engine satisfies the query using some SQL extension like DB2 "WAITFORDATA" or TiDB @@tidb_read_staleness or MemSQL columnstore_latency and/or similar signalling for performance-vs-cost preference.
And a common permissioning/datasharing layer so I can share data to external and internal parties who can in turn bring their own compute to make their own latency choices.
gregw2|3 months ago
And a common permissioning/datasharing layer so I can share data to external and internal parties who can in turn bring their own compute to make their own latency choices.
pgguru|3 months ago
That said, don't sleep on the "this is awesome" parts in this project... my personal favorite is the automatic schema detection:
``` CREATE TABLE my_iceberg_table () USING iceberg WITH (definition_from = 's3://bucket/source_data.parquet'); ```
unknown|3 months ago
[deleted]