[jaynate]

It’s sort of difficult to understand why this is even a question - LLM-based / judgment dependent workflows vs script-based / deterministic workflows.

In mapping out the problems that need to be solved with internal workflows, it’s wise to clarify where probabilistic judgments are helpful / required vs. not upfront. If the process is fixed and requires determinism why not just write scripts (code-gen’ed, of course).

[David]

This bothered me at first but I think it's about ease of implementation. If you've built a good harness with access to lots of tools, it's very easy to plug in a request like "if the linked PR is approved, please react to the slack message with :checkmark:". For a lot of things I can see how it'd actually be harder to generate a script that uses the APIs correctly than to rely on the LLM to figure it out, and maybe that lets you figure out if it's worth spending an hour automating properly.

Of course the specific example in the post seems like it could be one-shotted pretty easily, so it's a strange motivating example.

[pjm331]

It seems easier but in my experience building an internal agent it’s not actually easier long term just slow and error prone and you will find yourself trying to solve prompt and context problems for something that should be both reliable and instantaneous

These days I do everything I can to do straightforward automation and only get the agent involved when it’s impossible to move forward without it