top | item 45535026

(no title)

evnp | 4 months ago

Just curious, how do you go about making the test suite immutable? Was just reading this earlier today...

https://news.ycombinator.com/item?id=45525085

discuss

order

vidarh|4 months ago

Just don't give it write access, and rig it up so that you gate success on a file generated by running the test suite separate from the agent that it can't influence. It can tell me it has fixed things as much as it like, but until the tests actually passes it will just get told the problem still exists, to document the approach it tested and to document that it didn't work, and try again.

evnp|4 months ago

Appreciate the exposition, great ideas here. It's fascinating how the relationship between human and machine has become almost adversarial here!