[mikaelaast]

How do you verify the code without actually looking at it?

[adamzwasserman]

Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer.

So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Verification without reading every line, but not "vibe checking" either.

[mikaelaast]

I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.

[moomoo11]

"Let's check that we can do X, Y, Z"

"Create documentation and then write tests"

a few moments later...

"There's a bug where we cannot do Y. Investigate the code and then let's discuss the best fix"

"Update the documentation and tests"

[raw_anon_1111]

How do you verify the compiler without looking at the assembled code? How do you verify code that links against binary libraries?

You run it and check for your desired behavior.

[giantg2]

Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.

[mikaelaast]

(Those are hardly analogous comparisons to LLM generated code, are they?)

So you do a vibe check?