[kavok]

It’s amazing how often an LLM mocks or stubs some code and then writes a test that only checks the mock, which ends up testing nothing.

[Dig1t]

I have seen junior engineers do this on multiple occasions. This is why all code should be reviewed by experienced engineers, whether written by a human or an LLM.

[mapontosevenths]

You really do have to verify and validate the tests. Worse you have to constantly battle the thing trying to cheat at the tests or bypass them completely.

But once you figure that out, it's pretty effective.