re-run failed tests
The test runner should be able to keep track of failed tests (in a tmp file?) and provide an option to only re-run the previously failed tests. For example:
$./runTests --rerun_failed No previous results found, run all tests... test1: Passed test2: Failed (expected 3 got 2) test3: Failed (expected 1 got 3) There are 2 failing tests! # after changing some code and adding a new test ... $./runTests --rerun_failed test2: Passed test3: Failed (expected 1 got 4) test4 (new): Failed (expected 0 got 4) There are still 2 failing tests! # after changing code some more... $./runTests --rerun_failed test3: Passed test4: Passed Repeat remaining tests... test1: Passed test2: Passed All tests passed.
For this we have to agree on a format to tell test programs which tests to run. I was thinking of a simple command line option
--exclude_test .... Excluding tests makes more sense if we assume that new tests could be added between runs.