[elham]

Hey! Great question we get a lot. We've come from/talked to a lot of companies that do what you described with stuff like timescale and influxdb. They're useful tools and support a breadth of applications. We thought by building one to specifically leverage the read/write patterns you'd expect with sensor-heavy systems, we could achieve better data throughput and thus better enable real-time applications. For example, we've been able to get 5x write performance for sensor data on our DB compared to influxDB.

In general, having built out the core DB, it has been valuable in allowing us to expand to the other useful features such as being able to write commands out to hardware at sufficient control loop frequencies or create smooth real-time visualizations.

The other thing we think is really powerful is having a more integrated tool for acquiring, storing, and processing sensor data & actuating hardware. One common issue we experienced was trying to cobble together several tools that weren't fully compatible - creating a lot of friction in the overall control and acquisition workflow. We want to provide a platform to create a more cohesive but extensible system and the data storage aspect was a good base to build that off of.

[kpmcc]

Thanks for the reply! That all makes sense, and I can totally relate to the "cobbling together several tools that weren't fully compatible" experience. There's enough complexity with having to support or integrate sensors/actuators with a variety of industrial networking protocols. Anything to simplify the software portion of the system would go a long way. Excited to dig into this a bit more, best of luck with ongoing development!

[elham]

Thank you! Happy to get any feedback after a deeper look

[btown]

Did you build on any low-level libraries like RocksDB for data persistence etc.? Or did you fully hand-roll the database? Curious about the tradeoffs there nowadays.

[embonilla]

The core time series engine, called cesium (https://github.com/synnaxlabs/synnax/tree/main/cesium), is written completely from scratch. It's kind of designed as a "time-series S3", where we store blobs of samples in a columnar fashion and index by time. We can actually get 5x the write performance of something like InfluxDB in certain cases.

For other meta-data, such as channels, users, permissions, etc. we rely on CockroachDB's Pebble, which is a RocksDB compatible KV store implemented in pure go.