[Dig1t]

Only if they are supremely lazy. It’s possible to use these tools in a diligent way, where you maintain understanding and control of the system but outsource the implementation of tasks to the LLM.

An engineer should be code reviewing every line written by an LLM, in the same way that every line is normally code reviewed when written by a human.

Maybe this changes the original argument from software being “free”, but we could just change that to mean “super cheap”.

[collinvandyck76]

There's a pretty big difference between the understanding that comes with reviewing code versus writing it, for most people I think.

[macintux]

Definitely true for me. What’s particularly problematic is code I need to review but can’t effectively test due to environmental challenges.

[mapontosevenths]

> An engineer should be code reviewing every line written by an LLM,

I disagree.

Instead, a human should be reviewing the LLM generated unit tests to ensure that they test for the right thing. Beyond that, YOLO.

If your architecture makes testing hard build a better one. If your tests arent good enough make the AI write better ones.

[jddj]

The venn diagram for "bad things an LLM could decide are a good idea" and "things you'll think to check that it tests for" has very little overlap. The first circle includes, roughly, every possible action. And the second is tiny.

Just read the code.

[sarchertech]

There’s no way you or the AI wrote tests to cover everything you care about.

If you did, the tests would be at least as complicated as the code (almost certainly much more so), so looking at the tests isn’t meaningfully easier than looking at the code.

If you didn’t, any functionality you didn’t test is subject to change every time the AI does any work at all.

As long as AIs are either non-deterministic or chaotic (suffer from prompt instability, the code is the spec. Non determinism is probably solvable, but prompt instability is a much harder problem.

[kavok]

It’s amazing how often an LLM mocks or stubs some code and then writes a test that only checks the mock, which ends up testing nothing.

[vips7L]

The majority of devs I meet are extremely lazy. It’s why so many people are outsourcing their jobs to Claude.