[jdwithit]

Yes, regression testing--making sure bugs you've fixed don't return--is a standard part of QA. I did volunteer QA for Mozilla in college a good 20 years ago (god that number is horrifying) and they had an ever-growing suite of regression tests. Mostly for rendering/layout or JavaScript engine bugs, since part of reproducing and proving you'd fixed those was creating a minimal test case. Which you could then easily throw into the build pipeline.

Bugs are a fact of life, but burning time and money to fix them only to have them return is the worst case scenario. Organizations that care about quality are definitely investing in regression testing. Unfortunately a whole lot of orgs give QA zero respect and offshore it to the lowest bidder, if they do it at all. It's absolutely insane to me that Apple wouldn't have regression tests for jail breaks, some of the most high profile bugs in history.

You can fairly criticize Mozilla for a number of things these days. But they had a very robust QA and CI/CD setup in the early 2000s with tools like Tinderbox and Bugzilla. When DevOps came around and popularized it I was like wait, people weren't already doing this stuff??? Turned out I had been living in a bubble and that was not the norm at all.

[bradknowles]

Many years ago, I did a six month contract for Apple Retail Software Engineering, to deliver a Jenkins CI/CD system for the code that was used to communicate with the employees in the stores, to allow those employees to communicate with each other, to deliver training to them, etc….

There were multiple major components. There was the back-end server system that ran on Linux. There was the content creation system that ran on MacOS. There was the end-user clients that ran on iOS and iPadOS. And there was an extensive array of QA processes that they ran.

I ended up making minor changes to the code base for each of those components, so that they could build on the Jenkins server that was running underneath my desk (on an old Mac Pro server that had been lying around).

And I can tell you that they had extensive regression tests — as of the time I was there, over five thousand of them. Those took a really long time to run, which is why they needed the Jenkins server instead of doing this stuff on their laptops.

Now, I can’t speak for developers anywhere else at Apple, but I believe that they are well acquainted with the concept of regression testing.

[gcau]

I think they are referring to secretly regression testing other peoples code (to check if patched exploits become exploitable again).