Akka focuses on enterprise agentic with a focus on creating certainty and solving scale problems. We have a customer, Swiggy, which is >3M inferences per second for a blended set of models, both ML and LLMs, with a p99 latency of roughly 70ms.
This level of throughput is achieved by including memory database within the agentic process and then the clustering system automatically shards and balances memory data across nodes with end user routing built in. Combined with non-blocking ML invocations with back pressure you get the balance for performance.
The framework itself is super low overhead. You can deploy this anywhere, and if you deploy to inngest.com the P99 latency of starting agents is sub-50ms (and you can also realtime stream steps, tools, or model responses to the browser).
One of the main differences is the DX — _how_ you define the agentic worklflows is far cleaner, so it's both faster to build and fast in production.
TylerJewell|11 months ago
This level of throughput is achieved by including memory database within the agentic process and then the clustering system automatically shards and balances memory data across nodes with end user routing built in. Combined with non-blocking ML invocations with back pressure you get the balance for performance.
tonyhb|11 months ago
One of the main differences is the DX — _how_ you define the agentic worklflows is far cleaner, so it's both faster to build and fast in production.