[traverseda]

>We did this so we can scale.

>Python async sucks

Python async may make certain types of IO-blocked tasks simpler, but it is not going to scale a web app. Now maybe this isn't a web app, I can't really tell. But this is not going to scale to a cluster of machines.

You need to use a distributed task queue like celery.