top | item 41536898

(no title)

lor_louis | 1 year ago

I've worked on a database that considered FSST as a way to query over compressed log lines. We found that the compression ratio was highly dependent on how repetitive the data was. In the end segmenting by service (Apache, our go stuff, our rust stuff, etc) yielded pretty good results and log lengths of ~200 bytes were pretty well compressed.

We ended up not using it in production because the worst cases were absolutely terrible compared to our dumber skippable zstd.

discuss

order

No comments yet.