Tagged: 

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #33923
    Alex
    Participant

    Please explain more particularly a meaning of indicators in analysis of tests.

    Tests: 293
    Coverage: 100%
    T-sets covered / total (uncovered): 21424 / 21424 (0)

    Is T-sets a complete number of all various combinations of all parameters?
    How did you figure out that 293 tests covers all necessary situations?
    How do you evaluate for example that 16 test-cases in this situation will cover 82% of functionality?

    Thank you.

    #33928

    Hi Alex!

    > Is T-sets a complete number of all various combinations of all parameters?

    Yes, you are right: A t-set is an interaction: For example, if we have Parameter 1 with values A and B and Parameter 2 with values C and D, all the (pairwise) t-sets are: {A, C}, {A, D}, {B, C} and {B, D}. Notice that {A, B} and {C, D} are illegal and invalid t-sets.

    In the numbers you see (21424 / 21424 (0)), the first number is the number of t-sets covered by the tests. The second number is the total number of legal and valid t-sets. The third number is the difference between those numbers: The uncovered t-sets. If the coverage is 100% the number of uncovered t-sets is 0.

    We have planned to add some explanation of those numbers in addition to adding other interesting metrics.

    > How did you figure out that 293 tests covers all necessary situations?

    It is basically a greedy set cover algorithm. For an overview, you can read this paper:

    http://martinfjohansen.com/papers/Johansen2012.pdf

    The Pairwiser algorithm is based on this algorithm with some significant advancements that makes it several orders of magnitudes faster.

    > How do you evaluate for example that 16 test-cases in this situation will cover 82% of functionality?

    It is done using the algorithm and associated algorithms mentioned in the previous answer.

Viewing 2 posts - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.