top | item 46508422

(no title)

antonok | 1 month ago

FlatBuffers was definitely the majority of the improvements here!

On 64-bit systems, pointers themselves can really start to take up a lot of memory (especially if you multiply them across 100k+ adblock filters). Switching to array indices instead of pointers saves a lot of memory that's otherwise wasted when you don't need to address the entire possible memory space.

discuss

order

ThierryBuilds|1 month ago

Insightful. Many thanks.

Flat buffers is know to bloat client code. Was any trick used to mitigate that?

never_inline|1 month ago

I guess code bloat is proportional to schema complexity, and performance improvement is proportional to volume, so in ad blocker with many large block lists the latter dominates.

jeffreygoesto|1 month ago

The biggest improvement for us was deduplication by using generators an referencing already emitted objects. Don't run flatc on a JSON, it doesn't do that.