How does covrel compare with traditional white-box coverage-based selection heuristics?
We performed further evaluations to investigate the effectiveness of covrel when compared with traditional white-box coverage-based selection. For doing so, we adopted the greedy total and greedy additional heuristics as they are often considered the baseline for coverage-based selection approaches due to their high cost-effectiveness ratio.
In the tables below, we report the average number of test cases required to achieve the maximum attainable reliability by the different approaches. For interpreting the results the lower the number, the better. The cases in which the traditional coverage-based selection approach was not able to reveal all the faults is represented by a dash (–). For ease of readability we highlight in bold the cases in which covrel performed better than the greedy heuristics.
Greed total heuristic
Defaut fault-matrix (all faults)
Average number of test cases required by covrel and the greedy total heuristic to achieve the maximum attainable reliability (all faults)
"Hard" fault-matrix (hard-to-find faults)
Average number of test cases required by covrel and the greedy total heuristic to achieve the maximum attainable reliability (hard-to-find faults)
Greed additional heuristic
Defaut fault-matrix (all faults)
Average number of test cases required by covrel and the greedy total heuristic to achieve the maximum attainable reliability (all faults)
"Hard" fault-matrix (hard-to-find faults)
Average number of test cases required by covrel and the greedy total heuristic to achieve the maximum attainable reliability (hard-to-find faults)