top | item 44977461

(no title)

stillpointlab | 6 months ago

I'm still calibrating myself on the size of task that I can get Claude Code to do before I have to intervene.

I call this problem the "goldilocks" problem. The task has to be large enough that it outweighs the time necessary to write out a sufficiently detailed specification AND to review and fix the output. It has to be small enough that Claude doesn't get overwhelmed.

The issue with this is, writing a "sufficiently detailed specification" is task dependent. Sometimes a single sentence is enough, other times a paragraph or two, sometimes a couple of pages is necessary. And the "review and fix" phase again is totally dependent and completely unknown. I can usually estimate the spec time but the review and fix phase is a dice roll dependent on the output of the agent.

And the "overwhelming" metric is again not clear. Sometimes Claude Code can crush significant tasks in one shot. Other times it can get stuck or lost. I haven't fully developed an intuition for this yet, how to differentiate these.

What I can say, this is an entirely new skill. It isn't like architecting large systems for human development. It isn't like programming. It is its own thing.

discuss

order

scuff3d|6 months ago

This is why I'm still dubious about the overall productivity increase we'll see from AI once all the dust settles.

I think it's undeniable that in narrow well controlled use cases the AI does give you a bump. Once you move beyond that though the time you have to spend on cleanup starts to seriously eat into any efficiency gains.

And if you're in a domain you know very little about, I think any use case beyond helping you learn a little quicker is a net negative.

jmvldz|6 months ago

"It isn't like programming. It is its own thing."

You articulated what I was wrestling with in the post perfectly.

bdangubic|6 months ago

It isn't like programming. It is its own thing.

Absolutely. And what I find fascinating that this experience is highly personal. I read probably 876 different “How I code with LLMs” and I can honestly say not a single thing I read and tried (and I tried A LOT) “worked” for me…

rootnod3|6 months ago

According to most enthusiasts of LLM/agentic coding you are just doing it wrong then.

kace91|6 months ago

>I haven't fully developed an intuition for this yet, how to differentiate these.

The big issue is that, even though there is a logical side to it, part is adapting to a close system that can change under your feet. New model, new prompt, there goes your practice.

rcarr|6 months ago

For the longer ones, are you using AI to help you write the specs?

stillpointlab|6 months ago

My experience is: AI written prompts are overly long and overly specific. I prefer to write the instructions myself and then direct the LLM to ask clarifying questions or provide an implementation plan. Depending on the size of change I go 1-3 rounds of clarifications until Claude indicates it is ready and provides a plan that I can review.

I do this in a task_descrtiption.md file and I include the clarifications in its own section (the files follow a task.template.md format).

JeremyNT|6 months ago

> What I can say, this is an entirely new skill. It isn't like architecting large systems for human development. It isn't like programming. It is its own thing.

It's management!

I find myself asking very similar questions to you: how much detail is too much? How likely is this to succeed without my assistance? If it does succeed, will I need to refactor? Am I wasting my time delegating or should I just do it?

It's almost identical to when I delegate a task to a junior... only the feedback cycle of "did I guess correctly here" is a lot faster... and unlike a junior, the AI will never get better from the experience.