[sll22]

You say this isn't a build system. Is there a build system it is designed to work with? I'm not sure I understand the point of separating the test runner from the build system. Is it to allow any build system to work with it?

[1024bees]

Smelt is build system agnostic, but it seeks to be the "invoker" so to speak -- we've used it with a many build systems including make, cmake and bazel.

There are a few reasons to separate build and test systems:

build systems struggle to express constrained random testing, and some build systems even struggle to express directed testing sweeps, which are common patterns in design verification.

The other reason is that testing is often treated as a "leaf node" in a build graph, and it's not possible to describe tests depending on other tests.

Overall, testing is a different problem than building, and circuit design often requires complex testing flows that build systems aren't designed for.

[forgotmyacct2]

Agreed. One of the main properties of a (good) build system, is very aggressive caching. This only works if your build steps are deterministic. From my experience (testing firmware for embedded devices), even if your firmware simulation is totally deterministic, there is plenty of non-determinism created by UB, memory corruption, crazy things interns do with test scripts... So we don't typically cache our test results.

[sll22]

So do you support the APIs of different tools? Is there a list somewhere of told that are supported? Questa? VCS? Our so users need to figure that out?