top | item 36901602

(no title)

kastden | 2 years ago

Check out ProcessPoolExecutor [0] (and ThreadPoolExecutor) too for an easy way to spin up a bunch of tasks and wait for them to complete.

[0] https://docs.python.org/3/library/concurrent.futures.html#pr...

discuss

order

evomassiny|2 years ago

Agreed ! Plus, the ability submit a bunch of tasks, and to block until _one_ task completed (akin to `epoll_wait` of tokio's `select`) makes it quite useful. I don't know of a use case or `mutiprocessing.Pool` which is not covered by `concurrent.futures.ProcessPoolExecutor`; so I wonder why both exist

hangonhn|2 years ago

In general, the Executor or even Queue abstractions are much better and safer ways of doing multithreading and multiprocessing. These days, I rarely ever directly create threads. It's fine for situations when you must but parallelizing work can usually be done better, safer, and easier with executors.

gpderetta|2 years ago

Coincidence: I literally read that doc-page and wrote some ThreadPoolExecutor code 5 minutes ago to workaround the lack of a specific async operation in asyncio.