top | item 43367058

(no title)

diroussel | 11 months ago

It's great that they added iceberg support I guess, but it's a shame that they also removed S3 Select. S3 Select wasn't perfect. For instance the performance was no where near as good as using DuckDB to scan a parquet file, since duck is smart, and S3 Select does a full table scan.

But S3 Select is nearly way cheaper that the new iceberg support. So if your needs are only for reading one parquet snapshot, we no need to do updates, then this change is not welcome.

Great article though, and I was pleased to see this at the end:

> We’ve invested in a collaboration with DuckDB to accelerate Iceberg support in Duck,

discuss

order

No comments yet.