top | item 42829082

(no title)

pammf | 1 year ago

Iceberg has the hdfs catalog, which also relies only on dirs and files.

That said, a catalog (which Delta also can have) helps a lot to keep things tidy. For example, I can write a dataset with Spark, transform it with dbt and a query engine (such as Trino) and consume the resulting dataset with any client that supports Iceberg. If I use a catalog, all happens without having to register the dataset location in each of these components.

discuss

order

No comments yet.