re-run failed tests
The test runner should be able to keep track of failed tests (in a tmp file?) and provide an option to only re-run the previously failed tests. For example:
$./runTests --rerun_failed
No previous results found, run all tests...
test1: Passed
test2: Failed (expected 3 got 2)
test3: Failed (expected 1 got 3)
There are 2 failing tests!
# after changing some code and adding a new test ...
$./runTests --rerun_failed
test2: Passed
test3: Failed (expected 1 got 4)
test4 (new): Failed (expected 0 got 4)
There are still 2 failing tests!
# after changing code some more...
$./runTests --rerun_failed
test3: Passed
test4: Passed
Repeat remaining tests...
test1: Passed
test2: Passed
All tests passed.
For this we have to agree on a format to tell test programs which tests to run. I was thinking of a simple command line option --exclude_test ...
. Excluding tests makes more sense if we assume that new tests could be added between runs.