gruuya | 3 years ago | on: Rust visitor pattern and efficient DataFusion query federation
gruuya's comments
gruuya | 3 years ago | on: ConnectorX: Accelerating Data Loading from Databases to Dataframes [pdf]
What really struck me here at first was how Pandas read_sql spends so little time on the actual query execution and data transfer, while client side processing is taking up the majority (~85%) of time.
It makes more sense though, once you realise that they're talking about unsaturated networks, and so they can focus on relatively simple optimisation techniques (e.g. query partitioning and zero-copy) to bring about significant speedup in data loading.
gruuya | 3 years ago | on: Graphene may have found its killer app
Graphene's cousin, carbon nanotubes, have been touted as a concrete-strengthening additive for some time now[1], but the downside seems to be that corrosion sensitivity is increased.[2] Curious to see whether graphene would really be immune to that as is claimed.
"And when, eventually, these things do reach the ends of their lives, if the graphene can be recovered and used again it would be a welcome boost to building a circular economy."
This just sounds silly/unfeasible.
[1] https://www.frontiersin.org/articles/10.3389/fmats.2022.8616... [2] https://www.nature.com/articles/s41598-019-42761-2
gruuya | 3 years ago | on: Compostable fungi-based replacement for styrofoam
Funny enough, my initial impression was that it looks tasty as it reminded me of Camembert cheese.
gruuya | 4 years ago | on: Ask HN: Experience using Zanzibar-styled authorization in production?
Nice article, thanks for sharing - good to read about some background here. Interesting that Tailscale basically describes Google Zanzibar but does not name it explicitly.
I'm aware of authzed, and have watched their PWL talk. Fwiw, I found their solution to be very compelling, however it just doesn't seem to work well for the problem I'm working on (basically requiring a sidecar service running in a Docker container, as opposed to a SaaS).
gruuya | 5 years ago | on: Show HN: Django SQL Sniffer – monitor SQL executed by a running Python process
Thanks!
The tool doesn't work on windows currently, but I'll prioritize adding support for it; I believe it should be a minor tweak.
page 1
Though, I imagine there's a region in the data size/network throughput/latency space where simply fetching the data and then doing analytics in ClickHouse is more performant than actually going for the pushdown.[3]
[1] https://www.splitgraph.com/blog/postgresql-fdw-aggregation-p... [2] https://www.splitgraph.com/blog/postgresql-fdw-aggregation-p... [3] https://duckdb.org/2022/09/30/postgres-scanner.html