[felipetrz]

For this use case it would be better to put the data in a shared SQLite database than relying on multiprocessing CoW.

Even accessing objects from the shared memory would cause the reference counter to increment and the data would be copied, causing a memory usage explosion.

[coldtea]

>For this use case it would be better to put the data in a shared SQLite database than relying on multiprocessing CoW

In Python yes. In Java you could take advantage of shared memory and get spared the overhead of SQLite.