[fenollp]

Were any thoughts given to a process dictionary implementation? One (maybe supervised) process per instance of the pd_ordset type. The process would only ever have one client so no locking needed. Only potential issue would be the cost of sending messages. I have seen this kind of solution bring 100x speed improvements to write once <GB strings storage over ETS tables. This could be a gen_server as well. I’m curious because the PDict is often frowned upon (especially vocalized on the mailing list) yet cannot be avoided in some parts of OTP.

Also looking at the code leads me to believe the benchmarking of the rust code was done without taking the usage of a NIF’s overhead, if there is any. Is this intended or am I missing something?

Last question: are the great tracing capabilities of the BEAM lost when using NIFs and would this be an issue at Discord scale?

Thanks for the interesting write up.