top | item 39748732

(no title)

overbytecode | 1 year ago

How do you decide when to reach for Polars vs DuckDB?

discuss

order

jasonjmcghee|1 year ago

In Python, I think of them as, DuckDB is for getting the data you want, in the form you want it, from a file / remote place. Polars is for when you want to do something with that data, like visualize it.

`duckdb.sql("SELECT foo, bar * 2 as bar_times_2 FROM ...").pl()` (now in polars format) -> other stuff

In Rust, it's a bit fuzzier to me, as DuckDB is a pretty heavy dependency. I'm looking more and more fondly at DataFusion.

nozzlegear|1 year ago

Do you mean Polars depends on/uses DuckDB pretty heavily in Rust? I'm only just now dabbling in Rust myself so I'm not familiar.

joelschw|1 year ago

Or you use Ibis and switch between the two at will!