top | item 45912738

(no title)

greggyb | 3 months ago

I'd argue pipes are even more powerful with multi-core machines. You get parallelism for absolutely free. No thread pools to manage, no async, no channels, no new abstractions. Just the same pipe that works in the same way it did 50 years ago.

But because each step of the pipeline is a separate process, you get the N executables of the pipeline all able to run on separate cores.

I am not trying to say that every workload will benefit from this, or that such coarse parallelism is optimal for all use cases. But the fact that it is free with no changes to the pipe, the pipeline, or any of the executables is incredible.

discuss

order

anthk|3 months ago

The parent commenter ignores what xargs can do.