top | item 41685302

(no title)

nbsande | 1 year ago

run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!

discuss

order

zo1|1 year ago

I've had plenty of success using run_in_executor in production. You basically just use the decorator, and it mostly just works.

sandeep1998|1 year ago

Which decorator?

noident|1 year ago

> but my use case was more making async code utlize the cpu better

But run_in_executor achieves that as well! If you use no-GIL Python and a thread pool (or GIL Python with a Process pool), you will utilize more CPU cores.

d0mine|1 year ago

run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).

noident|1 year ago

>run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code

That isn't true. If you use a ProcessPoolExecutor as the target instead of the default executor, you will use multiple processes in pure Python code.