top | item 28910086

(no title)

proddata | 4 years ago

What do yo mean by high-frequency data? 100Hz, 1KHz, 100KHz? For that kind of use cases many time-series DBs break apart. We have customers storing multiple millions of high frequency measurements per sec in arrays.

I would say, Postgres is not too storage efficient in itself for large amounts of data, especially if you need any sorts of indexes. Timescale basically mitigates that by automatically creating new table in the background ("chunks") and keeping individual tables small.

discuss

order

LoriP|4 years ago

TimescaleDB also implements compression. From the docs:

When compression is enabled, TimescaleDB converts data stored in many rows into an array. This means that instead of using lots of rows to store the data, it stores the same data in a single row. Because a single row takes up less disk space than many rows, it decreases the amount of disk space required, and can also speed up some queries.

(Timescale employee)

digbybk|4 years ago

Generally in the 100Hz or 200Hz range for the time being. What do you mean by break apart?

proddata|4 years ago

Not being able to keep up with the incoming data. But 100-200Hz I'd consider fine for most