> if you're not running the test-suite on pre-push your tests are slow
Of course tests are slow. If they're fast they're skipping something. You probably want to be skipping something most of the time, because most of the time your change shouldn't cause any side-effects in the system, but you definitely want a thorough test suite to run occasionally. Maybe before merge, maybe before release, maybe on some other schedule.
Each individual change might be fine in isolation but cause excessive memory pressure when accumulated with other changes, for example. Unit tests won't catch everything, integration & functional (hardware in-the-loop) tests are needed. I even sometimes have to run tests in a thermal chamber repeatedly to cover the whole -40-105°C temperature range, since the firmware I work on runs on hardware that allows that range & performance varies with temperature.
What is a good time to run slower tests? My full test suite takes around 4 minutes to run, and it’s trending upwards. I am a solo developer, so I run it pre-push.
And what about tests that only need to run intermittently, such as broken link checkers?
0123456789ABCDE|4 months ago
if you're not running auto-format on file-save, your auto-formatter is slow
if you're not running a code checker with auto-fix on pre-commit, your code checker is slow
if you're not running the test-suite on pre-push your tests are slow
if your tooling is slow you need to pick better tooling or make them fast
you want to keep that loop tight and active
SAI_Peregrinus|4 months ago
Of course tests are slow. If they're fast they're skipping something. You probably want to be skipping something most of the time, because most of the time your change shouldn't cause any side-effects in the system, but you definitely want a thorough test suite to run occasionally. Maybe before merge, maybe before release, maybe on some other schedule.
Each individual change might be fine in isolation but cause excessive memory pressure when accumulated with other changes, for example. Unit tests won't catch everything, integration & functional (hardware in-the-loop) tests are needed. I even sometimes have to run tests in a thermal chamber repeatedly to cover the whole -40-105°C temperature range, since the firmware I work on runs on hardware that allows that range & performance varies with temperature.
nicbou|4 months ago
And what about tests that only need to run intermittently, such as broken link checkers?
1718627440|4 months ago
Or you care about your formatting and don't want to throw them away.