Async Python is still confusing af - when do I need it, what happens under the hood, does it actually help with performance, sometimes the GIL comes into play and sometimes it doesn't, why do we ever use threads at all if there's a GIL, why is it called asyncio if we can use it for anything. My mind is kind of scattered and people seems to be using a lot of async Python for some reason.
I agree that especially within the standard context of python and its syntax, async seems weird (because it's sprinkled into an existing paradigm).
The best mental model for me always was to think: Here's an await, that means "interpreter, go and do something else that's currently waiting while this is not done yet."
And that's all about IO, because what you can wait on is essentially IO.
By the way, I really wish there was a better story to executors in async python. To think we still have the same queue/pickle-based multiprocessing to send async tasks to another core is kind of sad. Hoping for 3.12 and beyond there.
[edit] one really neat example that helped me get asyncio was Guido van Rossum's crawler in 500 lines of python [1]. A lot of the syntax is deprecated now, but it's still a great walk-through
It's great for when you can do concurrent I/O tasks. For example running a web backend, web scraping, or many API calls. FastAPI (and Starlette that it's built off of) is an async web framework and in my experience performs well.
Basically, your program will normally halt when doing I/O, and won't proceed until that I/O is done. During that halt, your program is doing nothing (no cpu being used). With asyncio, you can schedule multiple tasks to run, so if one is halted doing I/O, another can run.
Edit: And AFAIK, the GIL does not come into play at all with async. Only when multithreading.
It makes concurrent programming much simpler than using threads.
Very few locking and care is needed with asyncio, as opposed to using threads. Race conditions are basically not a thing if you write reasonably idiomatic code.
It might be (or not) faster than using threads, but that's not the main benefit in my view - this easiness of use is.
It's basically a fancy way to do epoll() around file descriptors, but hide the need to keep a main loop and state, and keep it hidden as functions that run in fake concurrency, stopping whenever they block, and being executed again when their file descriptor has activity.
It doesn't necessarily improve performance. It's just a much easier way to do non-blocking I/O (note that blocking and threaded I/O is easier to do but much much heavier).
In personal experience, when you write anything more complicated using threads in python, I end up writing lots of boilerplate code that I wished I could just shove somewhere - Events, 1 element Queues, ContextVar contexts, all of which don't benefit for being named and make appearance only in two spots in code. Async removed a lot of this for me while also unifying the future ecosystem and making my code more composable and easy to integrate.
When working in a fully async context, it becomes very logical and natural. A great example of this is the FastAPI web framework. You never write any top-level code, only functions that the framework calls, so you never have to deal with the event loop directly. You basically just sprinkle some async and await keywords around your IO bottlenecks and things suddenly run smoother.
> Async Python is still confusing af - when do I need it,
It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes)
(Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines)
> does it actually help with performance,
Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead.
> sometimes the GIL comes into play and sometimes it doesn't,
The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it.
It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library.
> why do we ever use threads at all if there's a GIL,
Threads exist because it was there before asyncio & event loops came into Python.
> why is it called asyncio if we can use it for anything.
Its name came from PEP 3156, proposing the asyncio library back in 2012.
As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to complete before it can do anything else. Making tasks asynchronous allows it to do other stuff while it waits for a task's request to complete, increasing CPU & I/O utilization.
> My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up?
Highly recommend this video from mcoding: It's fairly simple & goes through a sample implementation.
Async looks like parallelism, but its just smart scheduling. The core concept of async is saying "hey, im waiting for something to complete thats not under my control (like waiting for data on a socket to be able to be read), go ahead and do other things in the mean time".
If you ever coded in sockets in C (and its a good exercise to do so), you probably have at some point ran across `select` which is essentially a non blocking way to check which sockets have data available to read, and then sequentially read the data. This gives the ability for a program to appear parallelized in the sense that it can handle multiple client connections, but its not truly parallel. Different clients can be handled at different time depending on which order they connect, which is asynchronous in nature (versus processing each client in sequence and waiting on each one to connect and disconnect before moving on to the next one)
Async in Python is basically this concept, with a core fundamental feature of time limited execution. Functions can say that they are pausing for x seconds, allowing other functions to run, or functions can say that they give a certain function x seconds to run before resuming execution. If you async code (along with any library you may use) doesn't contain any sleeps or timeouts, its exactly equivalent to synchronous code (since the event loop never really recieves a message that it can suspend a routine or cancel it). With sleeps and timeouts, you gain control over things that can potentially block, both from a caller perspective of not having a function call block your own, and from a callee perspective of not making your function blocking.
The use case is for it is that it is good for I/O bound operations like Threading is, but with the addition that you don't have to worry about synchronization or race conditions, since by design your code will have predictable access patterns. The downside is that your code and any libraries that you use within your code has to be implemented as async libraries, and any library that is async has to have async wrappers around the calls to its methods, which in turn means that your entire code has to be async.
Threading with Python is generally not useful, as its not true parallelism because of GIL. GIL allows only one thread in Python to run. Threading is safer in Python because of this, however it obviously has drawbacks. In general its best used if you want asyncio like performance with a library that is not written with async, since GIL is smart enough to detect when a thread is waiting for input and switch context.
True parallelism in Python is achieved with multiprocessing, however the use case is a little different. Rather than spinning off processes, you generally launch a bunch of worker processes up front (to avoid the larger overhead), then use smart scheduling to distribute work between these processes. Here though you do have to worry about race conditions and synchronization, and use things like locks and mutexes.
> Clearly create_task is as close as you get to free in the Python world, and I would need to look elsewhere for optimizations. Turns out Textual spends far more time processing CSS rules than creating tasks (obvious in retrospect).
Takeaways:
1. Creating async tasks is cheap.
2. It is important to confirm intuitions, before acting on them.
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done."
If you can, best is to always spawn them in a task group (either using anyio or Python 3.11's task groups).
This prevents tasks from being garbage collected, but also prevents situations where components can create tasks that outlive their own lifetime. Plus, it's a saner approach when dealing with exception handling and cancellation.
Python is my strongest language. If you ask me to write some asynchronous code I will try my best not to write it in Python. Usually it's just not worth it.
Yeah, I did the exact same thing and got very similar results. Just to save people clicking through, the Go/goroutine version is 25x as fast as Python. :-)
I preferred gevent (it's been probably 10 years since I've used it.) Yes, you need a ton of monkey patching, etc... but it was less intrusive once you had everything set up. Sprinkling await and async everywhere always struck me as inelegant.
Your method of estimating instructions includes printing multiple lines which context switch to the kernel to perform IO. I'm not sure how an "instruction count" metric is useful anyway.
edit: I'm not actually sure you are counting the context switch, but I still don't think estimating instruction count that way is particularly useful.
I have been looking into the overhead if async on c++ and we found that the cost increases substantially when the function has a return value. It would be interesting to see if this is the case with python 's asyncio.
Seems pretty decent, with that it’s a real shame Python’s async ended up coroutine-based rather than task-based. Given the langage semantics it ends up being a lot of pain for fairly little gain at the end if the day.
Mostly a testament to how absurdly fast modern CPUs are despite Python itself being so slow.
/ Recent Python convert, in spite of the horrible general performance of the official implementation of the language. That sweet, sweet module library. Also, with Docker containers the deployment issues have been solved. It might be slow to execute but it's really efficient to develop with.
This has always been the case. Computing power doubles every few years, and it's cheap (probably less than 10% of your total project cost, unless you are doing highly specific stuff). It doesn't make sense to optimise for CPU cycles as the real bottleneck is usually developer efficiency, I/O, and UX.
I haven't used asyncio that much, certainly not in any serious sense, but wouldn't ContextVar lookups be a major factor of performance in serious asyncio code? Using tasks for things that aren't io-bound seems likely to give a false sense of performance superiority when doing basically nothing.
Why would you want a terminal emulator anywhere near python? Using python for lightweight system utility gui apps seems like using a hammer to screw in a nail. Yeah you can do it and modern hardware is fast enough that you probably won't care, but why??
Reading this is like reading early Renaissance alchemist arguing about how much mercury they need to combine with how much silver to create gold... This is so far gone I don't even know where to begin...
> It may be IO that gives AsyncIO its name, but Textual doesn't do any IO of its own.
So why on Earth are you using AsyncIO? You don't need it, if that's true...
> Those tasks are used to power message queues
How are your message queues not doing I/O? What on Earth are they doing then?
Needless to say that the whole benchmark is worthless because it never even initiates anything that would be involved when creating actual asynchronous I/O tasks...
----
I mean, I know, in Pythonland this is just your average Wednesday, but dear lord, if you don't visit that land all that often it shocks you more every time you do.
This reply strongly reads of 'im smarter than eveyone' and 'know everything.' Is it possible that the author is doing something that you don't know about? Such as writing and reading text to sockets (which would be I/O.) And is it possible they may have a reason to be using asyncio in Python (which uses a single main thread and hence needs something like asyncio for concurrency.)
>Needless to say that the whole benchmark is worthless
Overhead startup isn't worthless.
'I mean, I know, in Pythonland this is just your average Wednesday'
and here we go. The whole post was really just you trying to make out that you're superior to everyone else. In this case looking down on an entire ecosystem. Why? Python is one of the most popular languages. It has an elegant syntax and it's capable of solving most problems. Go jerk yourself off in private.
Python asyncio is actually a co-operative multitasking system, think real-time operating system type stuff. It doesn't do real concurrency but it gives you a nice interface for reasoning about tasks (think thread) while being able to explicitly control when/where they yield control of the event loop, how often they run, etc.
>How are your message queues not doing I/O?
We generally wouldn't consider in-process moving data around to be "I/O", now if you started interacting with an external database/file/pipe/MMAP-ed-file/etc than that would be "I/O".
>What on Earth are they doing then?
Tasks. Kind of like processes but lighter weight. Running an event loop, dispatching signals, that sort of thing. You can do all that by manually writing your own event loop but python's asyncio (IMO) makes it easier to reason about exactly when your yielding the event loop to some other task, and makes it easier to write code that can yield control of the event loop at arbitrary places. So like if you want to update a widget every 10 seconds you can write something like
while True:
await asyncio.sleep(10) #Other tasks can run during this 10 seconds sleep
#Update widget contents
Useful for stuff that needs to periodically poll data, like a process monitor, or even for just simple clock widgets.
---
As an aside that cooperative multitasking can be really nice in micropython, where you can write tight loops in straight assembly if you need to and still get a pleasant task interface for managing higher level tasks/threads. Combined with some interrupt handlers it makes a pretty elegant real-time-ish operating system. (You probably need to manually deal with garbage collections though)
I knew you would be here in the comments. Crabbone won't leave any chance he gets to shit and dump on Python any HN post he can.
Must be so frustrating, knowing that this silly language is one of the most populair programming languages in the world, despite its short comings, one of which is slower performance.
This isn't about python or any language. The extreme negative tone and sense of superiority seems to point to something else.
asyncio is really not all about I/O though, despite the name. You can just as easily use it for concurrency by interleaving tasks with each other. You can imagine multiple UI elements that all need to make progress, and using coroutines to have them cooperatively yield to each other and not just have one element block progress everywhere else.
It's just using asyncio as a task scheduler, nothing more, nothing less. Maybe when you visit Pythonland you might instead be shocked in a more positive way :-)
Asyncio is for cooperative multitasking. I/O is the most common use case but it's not the only one. They're using the event loop to schedule their GUI tasks.
Textual is a framework for building desktop apps.
I assume the message queues are in-memory structures used to pass messages between tasks, hence no I/O.
This is really fairly standard stuff.
I understand you may not be familiar with GUI software and/or the Python ecosystem but jumping straight to condescension when you don't understand something is not really a good attitude.
Message queues can be implemented in memory, so no network or disk I/O. In fact, it seems this project does use the built-in "queue" library: https://docs.python.org/3/library/queue.html
[+] [-] jstx1|3 years ago|reply
Any good resources to clear things up?
[+] [-] uniqueuid|3 years ago|reply
The best mental model for me always was to think: Here's an await, that means "interpreter, go and do something else that's currently waiting while this is not done yet."
And that's all about IO, because what you can wait on is essentially IO.
By the way, I really wish there was a better story to executors in async python. To think we still have the same queue/pickle-based multiprocessing to send async tasks to another core is kind of sad. Hoping for 3.12 and beyond there.
[edit] one really neat example that helped me get asyncio was Guido van Rossum's crawler in 500 lines of python [1]. A lot of the syntax is deprecated now, but it's still a great walk-through
[1] http://aosabook.org/en/500L/a-web-crawler-with-asyncio-corou...
[+] [-] philote|3 years ago|reply
Basically, your program will normally halt when doing I/O, and won't proceed until that I/O is done. During that halt, your program is doing nothing (no cpu being used). With asyncio, you can schedule multiple tasks to run, so if one is halted doing I/O, another can run.
Edit: And AFAIK, the GIL does not come into play at all with async. Only when multithreading.
[+] [-] 323|3 years ago|reply
Very few locking and care is needed with asyncio, as opposed to using threads. Race conditions are basically not a thing if you write reasonably idiomatic code.
It might be (or not) faster than using threads, but that's not the main benefit in my view - this easiness of use is.
[+] [-] scrlk|3 years ago|reply
[+] [-] bombolo|3 years ago|reply
It doesn't necessarily improve performance. It's just a much easier way to do non-blocking I/O (note that blocking and threaded I/O is easier to do but much much heavier).
[+] [-] loa_in_|3 years ago|reply
[+] [-] franga2000|3 years ago|reply
[+] [-] x-complexity|3 years ago|reply
It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes)
> what happens under the hood,
https://tenthousandmeters.com/blog/python-behind-the-scenes-...
(Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines)
> does it actually help with performance,
Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead.
> sometimes the GIL comes into play and sometimes it doesn't,
The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it.
It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library.
> why do we ever use threads at all if there's a GIL,
Threads exist because it was there before asyncio & event loops came into Python.
> why is it called asyncio if we can use it for anything.
Its name came from PEP 3156, proposing the asyncio library back in 2012.
https://peps.python.org/pep-3156/
As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to complete before it can do anything else. Making tasks asynchronous allows it to do other stuff while it waits for a task's request to complete, increasing CPU & I/O utilization.
> My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up?
Highly recommend this video from mcoding: It's fairly simple & goes through a sample implementation.
https://www.youtube.com/watch?v=ftmdDlwMwwQ
Also, this article:
https://realpython.com/async-io-python/
[+] [-] ActorNightly|3 years ago|reply
If you ever coded in sockets in C (and its a good exercise to do so), you probably have at some point ran across `select` which is essentially a non blocking way to check which sockets have data available to read, and then sequentially read the data. This gives the ability for a program to appear parallelized in the sense that it can handle multiple client connections, but its not truly parallel. Different clients can be handled at different time depending on which order they connect, which is asynchronous in nature (versus processing each client in sequence and waiting on each one to connect and disconnect before moving on to the next one)
Async in Python is basically this concept, with a core fundamental feature of time limited execution. Functions can say that they are pausing for x seconds, allowing other functions to run, or functions can say that they give a certain function x seconds to run before resuming execution. If you async code (along with any library you may use) doesn't contain any sleeps or timeouts, its exactly equivalent to synchronous code (since the event loop never really recieves a message that it can suspend a routine or cancel it). With sleeps and timeouts, you gain control over things that can potentially block, both from a caller perspective of not having a function call block your own, and from a callee perspective of not making your function blocking.
The use case is for it is that it is good for I/O bound operations like Threading is, but with the addition that you don't have to worry about synchronization or race conditions, since by design your code will have predictable access patterns. The downside is that your code and any libraries that you use within your code has to be implemented as async libraries, and any library that is async has to have async wrappers around the calls to its methods, which in turn means that your entire code has to be async.
Threading with Python is generally not useful, as its not true parallelism because of GIL. GIL allows only one thread in Python to run. Threading is safer in Python because of this, however it obviously has drawbacks. In general its best used if you want asyncio like performance with a library that is not written with async, since GIL is smart enough to detect when a thread is waiting for input and switch context.
True parallelism in Python is achieved with multiprocessing, however the use case is a little different. Rather than spinning off processes, you generally launch a bunch of worker processes up front (to avoid the larger overhead), then use smart scheduling to distribute work between these processes. Here though you do have to worry about race conditions and synchronization, and use things like locks and mutexes.
[+] [-] Hendrikto|3 years ago|reply
Takeaways:
1. Creating async tasks is cheap. 2. It is important to confirm intuitions, before acting on them.
[+] [-] uniqueuid|3 years ago|reply
Be careful to hold your references, because async tasks without active references will be garbage collected. I've been bitten by that in the past.
Long discussion here: https://bugs.python.org/issue21163
Docs: https://docs.python.org/3/library/asyncio-task.html#asyncio....
"Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done."
[+] [-] jonathan_s|3 years ago|reply
This prevents tasks from being garbage collected, but also prevents situations where components can create tasks that outlive their own lifetime. Plus, it's a saner approach when dealing with exception handling and cancellation.
[+] [-] mrnonchalant|3 years ago|reply
[+] [-] gmadsen|3 years ago|reply
[+] [-] mkl95|3 years ago|reply
[+] [-] philote|3 years ago|reply
[+] [-] HyperSane|3 years ago|reply
[+] [-] schmichael|3 years ago|reply
[+] [-] bob1029|3 years ago|reply
Edit: Updating per request below.
Tested on a TR2950x. I wonder if NUMA issues in my case or bad python version (3.9.4).
Python
.NET 6[+] [-] benhoyt|3 years ago|reply
[+] [-] brrrrrm|3 years ago|reply
[+] [-] seanw444|3 years ago|reply
[+] [-] emmelaich|3 years ago|reply
[+] [-] icedchai|3 years ago|reply
[+] [-] jimmylt|3 years ago|reply
See: https://gist.github.com/jimmy-lt/4a3c6ad9cab1545692e5a3fe971...
[+] [-] INTPenis|3 years ago|reply
I only ever write an asyncio daemon when I want to launch and manage the results of a bunch of Celery tasks.
Not speaking from some superior position of research here, I'm just saying what I prefer to use.
[+] [-] TickleSteve|3 years ago|reply
Each of those task_create calls is roughly 10,000,000,000 / 250,000 = 40,000 instructions.
Thats 40'000 instructions of pure overhead as it does not contribute to the task at hand (accidental complexity).
[+] [-] schmichael|3 years ago|reply
edit: I'm not actually sure you are counting the context switch, but I still don't think estimating instruction count that way is particularly useful.
[+] [-] Kab1r|3 years ago|reply
[+] [-] masklinn|3 years ago|reply
[+] [-] tpmx|3 years ago|reply
/ Recent Python convert, in spite of the horrible general performance of the official implementation of the language. That sweet, sweet module library. Also, with Docker containers the deployment issues have been solved. It might be slow to execute but it's really efficient to develop with.
[+] [-] selcuka|3 years ago|reply
[+] [-] johndubchak|3 years ago|reply
[+] [-] zomnoys|3 years ago|reply
[+] [-] nathanasmith|3 years ago|reply
[+] [-] jayde2767|3 years ago|reply
[+] [-] johndubchak|3 years ago|reply
[+] [-] SCUSKU|3 years ago|reply
100,000 tasks 184,167 tasks per/s
200,000 tasks 160,964 tasks per/s
300,000 tasks 165,278 tasks per/s
400,000 tasks 149,577 tasks per/s
500,000 tasks 160,593 tasks per/s
600,000 tasks 168,098 tasks per/s
700,000 tasks 161,837 tasks per/s
800,000 tasks 160,364 tasks per/s
900,000 tasks 149,479 tasks per/s
1,000,000 tasks 155,919 tasks per/s
[+] [-] l_theanine|3 years ago|reply
[+] [-] willm|3 years ago|reply
[+] [-] chinaman425|3 years ago|reply
[deleted]
[+] [-] t3pfaff|3 years ago|reply
[+] [-] crabbone|3 years ago|reply
> It may be IO that gives AsyncIO its name, but Textual doesn't do any IO of its own.
So why on Earth are you using AsyncIO? You don't need it, if that's true...
> Those tasks are used to power message queues
How are your message queues not doing I/O? What on Earth are they doing then?
Needless to say that the whole benchmark is worthless because it never even initiates anything that would be involved when creating actual asynchronous I/O tasks...
----
I mean, I know, in Pythonland this is just your average Wednesday, but dear lord, if you don't visit that land all that often it shocks you more every time you do.
[+] [-] Uptrenda|3 years ago|reply
>Needless to say that the whole benchmark is worthless
Overhead startup isn't worthless.
'I mean, I know, in Pythonland this is just your average Wednesday'
and here we go. The whole post was really just you trying to make out that you're superior to everyone else. In this case looking down on an entire ecosystem. Why? Python is one of the most popular languages. It has an elegant syntax and it's capable of solving most problems. Go jerk yourself off in private.
[+] [-] traverseda|3 years ago|reply
>How are your message queues not doing I/O?
We generally wouldn't consider in-process moving data around to be "I/O", now if you started interacting with an external database/file/pipe/MMAP-ed-file/etc than that would be "I/O".
>What on Earth are they doing then?
Tasks. Kind of like processes but lighter weight. Running an event loop, dispatching signals, that sort of thing. You can do all that by manually writing your own event loop but python's asyncio (IMO) makes it easier to reason about exactly when your yielding the event loop to some other task, and makes it easier to write code that can yield control of the event loop at arbitrary places. So like if you want to update a widget every 10 seconds you can write something like
Useful for stuff that needs to periodically poll data, like a process monitor, or even for just simple clock widgets.---
As an aside that cooperative multitasking can be really nice in micropython, where you can write tight loops in straight assembly if you need to and still get a pleasant task interface for managing higher level tasks/threads. Combined with some interrupt handlers it makes a pretty elegant real-time-ish operating system. (You probably need to manually deal with garbage collections though)
[+] [-] louwrentius|3 years ago|reply
Must be so frustrating, knowing that this silly language is one of the most populair programming languages in the world, despite its short comings, one of which is slower performance.
This isn't about python or any language. The extreme negative tone and sense of superiority seems to point to something else.
What is it?
[+] [-] vore|3 years ago|reply
It's just using asyncio as a task scheduler, nothing more, nothing less. Maybe when you visit Pythonland you might instead be shocked in a more positive way :-)
[+] [-] greenshackle2|3 years ago|reply
Textual is a framework for building desktop apps.
I assume the message queues are in-memory structures used to pass messages between tasks, hence no I/O.
This is really fairly standard stuff.
I understand you may not be familiar with GUI software and/or the Python ecosystem but jumping straight to condescension when you don't understand something is not really a good attitude.
[+] [-] philote|3 years ago|reply