It is a great step, but in my testing with the new JSON type if you use beyond 255 unique json locations/types (255 max_dynamic_types in their config) you will fall back to much worse performance for certain queries and aggregations. This is quite easy to hit with some of the suggestions in this blog post, especially if you are designing for multi-tenant use.
A small question on the schema, I noticed that you have only “_now” as the Order By (so should just use that for the primary key). Do you expect any cross tenant queries?
Just my feeling would be that I’d add the tenant ID before the timestamp as it should filter the parts more effectively
JSON column type in ClickHouse [1] looks promising, since it allows storing wide events with arbitrary sets of fields. This feature is still in beta. Let's see how it will evolve.
maxmcd|1 year ago
For this clickhouse wide event lib I'm working on (not worth anyones time atm) I am still using this schema https://www.val.town/v/maxm/wideLib#L34-39 (which is from a Boris Tane talk https://youtu.be/00gW8txIP5g?t=801) for good multi-tenant performance.
I hope clickhouse performance here can still be vastly improved, but I think it is a little awkward to get optimal performance with wide events today.
bonobocop|1 year ago
Just my feeling would be that I’d add the tenant ID before the timestamp as it should filter the parts more effectively
hipadev23|1 year ago
[1] https://clickhouse.com/blog/a-new-powerful-json-data-type-fo...
valyala|1 year ago
[1] https://clickhouse.com/docs/en/sql-reference/data-types/newj...
brunoqc|1 year ago