[jauco]

You could repeatedly have developers execute tasks with and without ai (two different tasks. Each subject does the same 2 tasks. Randomize order and what task is done with and what task is done without ai) and show significant differences in duration.

You’d need probably at least 30 to 40 people (there’s ways to estimate this, but this is gut feeling from years ago when I did studies like this)

It would take on a few flaws in tfa which has a low number of people who operate in a codebase that they know very well on issues that they self-selected (and thus already will have a (perhaps subconscious) strategy for).

That would help convincing skeptics.

[Jensson]

Didn't this study do that and proved the skeptics right? It is hard to convince skeptics when the skeptics are right.

[andrewstuart]

I have no interest in convincing skeptics. They’re welcome to do as they choose. Computing and programming should be about pleasing yourself and programming in the way you like.