All of our processing is done via Ray (https://www.ray.io/). Our early benchmarks are about 5k mesesages per second on a single 4 core VM, but we believe we can increase the with some more optimizations.
This bench mark was consuming a Google Cloud Pub/Sub stream and outputting to BigQuery.
dmatrixjsd|3 years ago