We do something similar in house where I work. Is it hard to onboard new customers? Since they make a special container for you they basically adopt your build system, which may be hard for them. Does this go beyond mutation testing?
With most of our customers, they don't actually have to change their build system at all. We take their normal CI products, plus a small amount of special configuration, and run them. This is actually a good thing because we're testing something very similar to what runs in production.
The kinds of state space exploration we do are a lot more general than mutation testing. Our current product does exploration by (1) varying the space of faults, packet delivery times, thread schedules, etc., and (2) driving a customer-provided pseudo-randomized workload. We have plans to make both these mechanisms much more expressive, powerful, and configurable; and we have longer-term plans to add entirely new kinds of testing to the same platform.
Disclosure: I'm one of the co-founders of Antithesis.
Thanks! Where does the "want" come from to compare your "got"s to? If the customer code handles errors gracefully, can you still surface undesirable behavior?
wwilson|1 year ago
The kinds of state space exploration we do are a lot more general than mutation testing. Our current product does exploration by (1) varying the space of faults, packet delivery times, thread schedules, etc., and (2) driving a customer-provided pseudo-randomized workload. We have plans to make both these mechanisms much more expressive, powerful, and configurable; and we have longer-term plans to add entirely new kinds of testing to the same platform.
Disclosure: I'm one of the co-founders of Antithesis.
ilzmastr|1 year ago