minpur's comments

minpur | 7 years ago | on: Query Parquet files in SQLite

We used to pump CSV files into BigQuery from remote locations. Data availability became a problem because of choppy/low bandwidth internet in many of the remote wind/solar sites we work in. We explored converting CSV’s to Parquet format mainly for the reduced size Parquet offers. BigQuery accepts Parquet files but still in beta. We then just zipped the CSV files which reduced the size to almost 1/8 and BigQuery accepts zipped files directly. Works like a charm, only downside is if your CSV files have zagged rows then errors are thrown up. You can change some settings and have BigQuery accept a certain number of zagged rows.

If you are considering only file sizes then i would recommend this zipping method. If you are more interested in query time capabilities/advantages then you must consider Parquet.

We are still experimenting with Parquet because our dataset keeps increasing by about 100GB everyday.

minpur | 7 years ago | on: “They’ll Just Make It Illegal”

There is a valid case where crypto will occupy the place of black-money in countries like India. There is a minimum rate fixed by the government, especially for land. But the market rate is (many times) a multiple of that rate. So when you buy land, you pay the minimum government rate (plus tax) so you can enter into ‘legally binding’ sale agreements. The rest you pay in crypto.

In other worlds fiat currency issued by governments will continue to the extent of the legal sanctity they provide to any transaction. The rest is crypto.

minpur | 7 years ago | on: What the Airline Knows About the Guy in Seat 14C

This is quite a generic rant about airlines trying to make sense of passenger data. Nothing related to seat 14C. I mostly pay more and reserve the 12/13 row seats (emergency seats) because of better leg space. I’ve never had a hostess budge me with personal details.
page 1