- This topic has 1 reply, 2 voices, and was last updated 5 years, 1 month ago by .
Viewing 2 posts - 1 through 2 (of 2 total)
Viewing 2 posts - 1 through 2 (of 2 total)
- You must be logged in to reply to this topic.
Tagged: analysis
Please explain more particularly a meaning of indicators in analysis of tests.
Tests: 293
Coverage: 100%
T-sets covered / total (uncovered): 21424 / 21424 (0)
Is T-sets a complete number of all various combinations of all parameters?
How did you figure out that 293 tests covers all necessary situations?
How do you evaluate for example that 16 test-cases in this situation will cover 82% of functionality?
Thank you.
Hi Alex!
> Is T-sets a complete number of all various combinations of all parameters?
Yes, you are right: A t-set is an interaction: For example, if we have Parameter 1 with values A and B and Parameter 2 with values C and D, all the (pairwise) t-sets are: {A, C}, {A, D}, {B, C} and {B, D}. Notice that {A, B} and {C, D} are illegal and invalid t-sets.
In the numbers you see (21424 / 21424 (0)), the first number is the number of t-sets covered by the tests. The second number is the total number of legal and valid t-sets. The third number is the difference between those numbers: The uncovered t-sets. If the coverage is 100% the number of uncovered t-sets is 0.
We have planned to add some explanation of those numbers in addition to adding other interesting metrics.
> How did you figure out that 293 tests covers all necessary situations?
It is basically a greedy set cover algorithm. For an overview, you can read this paper:
http://martinfjohansen.com/papers/Johansen2012.pdf
The Pairwiser algorithm is based on this algorithm with some significant advancements that makes it several orders of magnitudes faster.
> How do you evaluate for example that 16 test-cases in this situation will cover 82% of functionality?
It is done using the algorithm and associated algorithms mentioned in the previous answer.