[avdempsey]

We’ve been using temporal (the Python SDK) for some new projects at Internet Archive. It’s early days but we’re very excited. We run our own infrastructure, and we get more power-loss events or other intermittent issues than most. The durable execution of workflows that temporal promises seems like it was made for us. And once code is “temporalized” we get to eject a bunch of ad-hoc resiliency stuff into the sun, and what’s left is a lot clearer.

There’s a lot to learn with it. I’ve seen ramp-up take a few months per engineer, though we’re also making it a little harder on ourselves by self-hosting, being the early adopters on the Python SDK (Go, Java, and TypeScript are the most mature I think), and dealing with a mix of Python async and multiprocessing (a bunch of CPU bound activities in the mix). The docs are solid, and the team is responsive to community users.

[overbytecode]

Can I ask what issues or difficulties you faced going the self-hosting route? We’re a small outfit and we’re thinking of self-hosting temporal to save costs, would be interested to hear about your experience.

[avdempsey]

Our department is still in a world of ansible and VMs, so we can't yet take advantage of some of the work that's gone into making it easy to run in k8s. We're using Postgres for Temporal's persistence because we're already familiar with operating it (and it's pretty cool already you can have some choice in DB). We've hit a bug in the newest version of Temporal server that doesn't play nicely with pgbouncer in transaction pooling mode, but they've responded to our ticket and seem to have a solution. We're running on the previous version for now. Other than that, it's just been the up-front cost of building the ansible playbooks. We haven't pushed it to any kind of load limits yet, or built-out a real high availability deploy of it yet. Day to day operation at our utilization level has been no drama so far.