top | item 45151966

(no title)

golergka | 5 months ago

All E2E tests should run before deploy. Probably on every commit on develop branch, even. But there really is no need to run all E2E suite on every PR. In this case, the failure mode of this system, where PR automation failed to flag a breakage, is acceptable if it’s rare enough, so probabilistic solutions are OK.

discuss

jampa|5 months ago

We still run E2E tests before deployment, but running them on pull requests also eliminates the question: "We want to deploy, but have a bug. Which PR caused it? and how do we fix it?" This approach essentially saves you from having to perform a git bisect and engineers getting blocked because there is a bug unrelated to their task.

golergka|5 months ago

Just run them on every commit in develop branch after the PR is merged. Then you can always roll back or revert the merge and leave it to the PR author to fix it.

catlifeonmars|5 months ago

Yeah this. You can commit optimistically and worst case can always revert a handful of commits eg on the nightly builds running e2e and manually root cause the breaking commit(s).