Stepped is a Rails engine, extracted out of Envirobly where it powers tasks like application deployment, that involve complex, out-of-the-band tasks like DNS provisioning, retries, waiting for instances to boot, running health checks and all the fun stuff of a highly distributed networked system.
Nice work. Thanks for sharing it! I've been thinking about using something like this for LLM agent workflows - the outbound action pattern would work well for tool calls that need to wait on external APIs.
I'm working on DSPy.rb [1] and this could pair nicely for multi-step reasoning chains.
Not sure how async gem would fit into this. The backbone of Stepped Actions is ActiveJob which handles execution. Where do you think this would fit in?
Congratulations on shipping this, I’m sure folks will find it useful!
The rails native way to do this is to track state in a db row and queuing “next step” jobs as the data changes. This can get verbose especially for smaller pass/fail workflows. However, I find this works better (not worse imo) in more complex workflows as the state is tracked, queryable, can be surfaced in UIs, and resumed “manually” in the event of an outage.
Nice. I have a simple system for typescript [1] where you can string tasks together like:
import { Workflow } from "@workglow/task-graph";
const workflow = new Workflow();
workflow
.DownloadModel({
model: ["onnx:Xenova/LaMini-Flan-T5-783M:q8", "Universal Sentence Encoder"],
})
.TextEmbedding({
text: "The quick brown fox jumps over the lazy dog.",
});
await workflow.run();
It automatically caches results of steps if you say `new Workflow({outputCache});`
PS: the example above uses a local onnx model (it switches on model name) which is a good candidate for caching, but can be anything.
You can play around writing this way if you open the console at the web example [2] which has fun console formatters not enough people know about or use in DevTools. You can write code in the console and the example rewrites it as a json format in the web page (and visa-versa).
Or just use the free web app with local user account and local models to play around with. [3]
I have implemented (or managed a team that implemented) this concept four separate times as an internal library somewhere. Each one included slightly different affordances and addressed a few different concerns that stem from the domain that it was built for.
Props for extracting it and offering it up as a library. I'll be interested to compare it to the implementations I've seen, and see what you've added that I've not seen before.
Seems pretty similar to https://github.com/radioactive-labs/chrono_forge which is what I found when I typed in "rails durable execution patterns" into Google. Have you seen this and if so, how do you think it compares?
This looks useful. I've been exploring similar durable execution patterns in Go recently to avoid the complexity of Temporal for smaller workflows.
How does stepped_actions handle the state between steps? Does it persist to the DB after every single action to handle crash recovery, or is it more optimistic?
Yes, state is persisted to DB upon every change. Action exceptions are handled gracefully and natural part of the system, they simply fail the action. Crash recovery is build-in thanks to checksums and ActiveJob, if you're using the right adapter, like GoodJob or SolidQueue where crash recovery is guaranteed.
vicentereig|2 months ago
I'm working on DSPy.rb [1] and this could pair nicely for multi-step reasoning chains.
Curious - any plans for async gem support?
[1] https://oss.vicente.services/dspy.rb/
klevo|2 months ago
reedlaw|2 months ago
deedubaya|2 months ago
The rails native way to do this is to track state in a db row and queuing “next step” jobs as the data changes. This can get verbose especially for smaller pass/fail workflows. However, I find this works better (not worse imo) in more complex workflows as the state is tracked, queryable, can be surfaced in UIs, and resumed “manually” in the event of an outage.
klevo|2 months ago
sroussey|2 months ago
import { Workflow } from "@workglow/task-graph"; const workflow = new Workflow(); workflow .DownloadModel({ model: ["onnx:Xenova/LaMini-Flan-T5-783M:q8", "Universal Sentence Encoder"], }) .TextEmbedding({ text: "The quick brown fox jumps over the lazy dog.", }); await workflow.run();
It automatically caches results of steps if you say `new Workflow({outputCache});`
PS: the example above uses a local onnx model (it switches on model name) which is a good candidate for caching, but can be anything.
You can play around writing this way if you open the console at the web example [2] which has fun console formatters not enough people know about or use in DevTools. You can write code in the console and the example rewrites it as a json format in the web page (and visa-versa).
Or just use the free web app with local user account and local models to play around with. [3]
[1] https://github.com/workglow-dev/workglow
[2] https://workglow-web.netlify.app/
[3] https://workglow.dev
chao-|2 months ago
Props for extracting it and offering it up as a library. I'll be interested to compare it to the implementations I've seen, and see what you've added that I've not seen before.
silasb|2 months ago
klevo|2 months ago
moh_quz|2 months ago
How does stepped_actions handle the state between steps? Does it persist to the DB after every single action to handle crash recovery, or is it more optimistic?
Good luck with the launch
nzoschke|2 months ago
https://github.com/dbos-inc/dbos-transact-golang
I have River on my list to try someday too
https://riverqueue.com/pro
klevo|2 months ago
dzonga|2 months ago