[psyonity]

To me this feels like it would test what the code does, not what the code is _supposed_ to do.

Tests should in my opinion validate the intent behind code, not the actual function itself.

[twobitshifter]

Right it is good as a “regression” test framework. AI isn’t needed for this and there are things like pyarttest that give you the same regression benefit. - they don’t validate that the initial code was correct.

[bobismyuncle]

I think you’re spot on that currently it can only test what the code actually does.

That’s probably why they specify that it’s for regression tests which are meant to do exactly that - test that future changes to the code do not change the current behavior unintentionally

[gavmor]

That's all well and good when you're test-driving, but when you're spelunking into a legacy codebase it can be awfully hard to identify "intent," so it's good to know when you've changed any observable behavior.