[t_mann]

> The ‘give up after ten attempts’ threshold aims to prevent Claude from wasting tokens when further progress is unlikely. It was only partially successful, as Claude would still sometimes make dozens of attempts.

Not what I would have expected from a 'one-shot'. Maybe self-supervised would be a more suitable term?

[wavemode]

"one-shot" usually just means, one example and its correct answer was provided in the prompt.

See also, "zero-shot" / "few-shot" etc.

[simonw]

I've seen one-shot used to mean two different things in LLMs:

1. Getting an LLM to do something based on a single example

2. Getting an LLM to achieve a goal from a single prompt with no follow-ups

I think both are equally valid.

[t_mann]

The article says that having decompiled some functions helps with decompiling others, so it seems like more than one example could be provided in the context. I think the OP was referring to the fact that only a single prompt created by a human was used. But then it goes off into what appears to be an agentic loop with no hard stopping conditions outside of what the agent decides.

We're essentially trying to map 'traditional' ML terminology to LLMs, it's natural that it'll take some time to get settled. I just thought that one-shot isn't an ideal name for something that might go off into an arbitrarily long loop.

[voiper1]

I definitely didn't expect one-shot to mean "let it run itself in an indefinite loop"

[johnfn]

One shot just means one prompt. What Claude decides to do during that prompt is up to it.

[hombre_fatal]

Meh, the main idea of one-shot is that you prompted it once and got a good impl when it decided it was done. As opposed to having to workshop yourself with additional prompts to fix things.

It doesn't do it in one-shot on the GPU either. It feeds outputs back into inputs over and over. By the time you see tokens as an end-user, the clanker has already made a bunch of iterations.