top | item 34742454 (no title) bigdubs | 3 years ago We use DuckDB extensively where I work (https://watershed.com), the primary way we're using it is to query Parquet formatted files stored in GCS, and we have some machinery to make that doable on demand for reporting and analysis "online" queries. discuss order hn newest dstroot|3 years ago Storing data in Parquet files and querying via DuckDB is fast and kind of magical. bigdubs|3 years ago Shockingly fast and nice and having the intermediate files be immutable is super nice. load replies (1) mritchie712|3 years ago Do you load the Parquet files in duckdb or just query them directly? bigdubs|3 years ago We query them directly in most cases with the httpfs plugin, but for hot paths we fetch them and cache them on disk locally.
dstroot|3 years ago Storing data in Parquet files and querying via DuckDB is fast and kind of magical. bigdubs|3 years ago Shockingly fast and nice and having the intermediate files be immutable is super nice. load replies (1)
bigdubs|3 years ago Shockingly fast and nice and having the intermediate files be immutable is super nice. load replies (1)
mritchie712|3 years ago Do you load the Parquet files in duckdb or just query them directly? bigdubs|3 years ago We query them directly in most cases with the httpfs plugin, but for hot paths we fetch them and cache them on disk locally.
bigdubs|3 years ago We query them directly in most cases with the httpfs plugin, but for hot paths we fetch them and cache them on disk locally.
dstroot|3 years ago
bigdubs|3 years ago
mritchie712|3 years ago
bigdubs|3 years ago