[allreduce]

Usually it does not reduce latency but increases throughput.

Multithreading is an invaluable tool when actually using your computer to crunch numbers (scientific computing, rendering, ...).