[tiehuis]

This is a nice write-up. Since zig has quite extensive compile-time functionality, one drawback is it is often easy to miss untested branches, especially for non-native targets if they are not regularly tested. It would be quite beneficial to be able to combine coverage reports from different targets to get a full picture of the tested code.

I have previously used kcov [1] to perform coverage testing which due to zig having good DWARF debugging information worked with no fanfare. However, I'm not too familiar with the shortcomings of this method vs. the more fully instrumented binary approach provided.

[1] https://github.com/SimonKagstrom/kcov

[squeek502]

Cool, I wasn't aware of kcov. Looks like kcov sets breakpoints on every line to get its coverage information, which seems like it'd probably be comparable to what callgrind is doing. The difference would be in execution speed, and after testing kcov, it looks like it wins that handily:

- Running the zig std lib tests normally takes ~5 seconds

- Running the zig std lib tests with callgrind takes ~2 minutes

- Running the zig std lib tests with kcov takes ~9 seconds

There are also slight disagreements in terms of the results:

- grindcov reports 43480 of 52586 lines executed or 82.68% coverage

- kcov reports 44103 of 52335 lines executed or 84.3% coverage

This difference might be down to kcov following child processes while grindcov doesn't, but I'm not totally sure.

kcov is probably the better choice overall.

[touisteur]

GNATCoverage (Ada-only) has interesting binary coverage capabilities. Qemu trace, valgrind instrumentation, nexus... If interested I built once A gdb-branch-trace script, a pintool to generate the trace or an Intel-PT-based prototype. Dyninst and dynamo rio too. Neat exercise