Continuous agents and what happens after Ralph Wiggum?

[waynenilsen]

Is anyone else doing the full software lifecycle for toy projects completely hands off the wheel? I have had Claude running in a Ralph like loop for over 15 hours unsupervised creating over 118 commits.

The technique works like this

while true: if tickets exist -> burn down the backlog by one ticket, exit if not -> figure out what feature would make sense to add next, create PRD and ERD, break down into tickets, exit

It did get stuck once due to tty issues related to running playwright in a non-tty environment but otherwise I have not had to manually step in.

I have it running in a droplet using systemd continuously.

Toy code the agent is creating is a multi-tenant todo kata. Here is the set of prompts:

https://github.com/waynenilsen/ralph-kata-2/tree/main/prompts

Anyone could make their own version of the same, these are just the set of prompts that work for me.

In 15 hours it created a full multi-tenant auth system from scratch and todos with assignees due dates, email reminders, tags and full text search. I created the first PRD by hand with something like "create a PRD for a multi-tenant todo system".

For anyone looking to do something similar, the e2e tests have played a critical role in closing the agent's loop with reality.

The age of programming with prompts is clearly arriving.

[ironbound]

You'd believe LLM's could reverse engineering software.. but this is not the case today

[codingdave]

> it created a full multi-tenant auth system from scratch

OK. And did that scratch auth system pass any level of security testing? If it did, great, that is worth talking about. But what I've seen generated by AI isn't anywhere near secure.

[waynenilsen]

i have seen the same, however, it can often easily find its own bugs when prompted to do so, in this case, with a ticket perhaps

the ticket burndown is a very nice feature because whenever you want to add a ticket it'll just pick it up and do its best

[waynenilsen]

> intent drifted

Indeed I believe this is probably the next thing to solve but even here I don't think it is out of reach. What we aught to be able to do is disconnect and make asynchronous the goals of the project with where we are. This, in normal software building, is encapsulated by the roadmap. I am building roadmapping prompts now and broadening the scope of the software development lifecycle even further to the encapsulate the roadmap as well which was previously out of scope for the experiment I am running now.

The prompts I am using now give the agent autonomy over 'make the next prd that makes sense' however I think it is a straightforward extension to add 'in the context of the @roadmap/ ' or similar with probably decent results.

Have you tried something similar?

Even without a roadmap the agent continues to do useful work over 24 hours in. You can see the commits and PRDs they really are quite sensible and I pulled and tested and everything really is working quite well. Frankly, I am shocked it is working at all. I have had to step in once or twice you definitely need to keep an eye on the logs every once in a while. Getting the loop booted up in a reliable way was the hardest part to be honest and even that was not terribly difficult.

https://github.com/waynenilsen/ralph-kata-2