Forum Replies Created

AuthorPosts

Hi Gordon, the Pairwiser server sitting behind the webinterface accepts general constrains written as propositional logic or CNF. This is not currently supported in the webinterface.
We could accomodate for it in the webinterface. Could you elaborate some more on your case?
Hi Bret, yes, you can now export the tests to Excel!
Hi Rajesh!
Have you taken a look at the two examples in our knowledge base?
Example 1: Classic Example
Example 2: Testing a WebsiteThese discuss the issues you mention in relation to two different examples.
The significance and applicability of 1, 2 and 3wise testing depends on the case you are applying it to. If you describe your testing problem a bit, I can describe the significance and applicability to it.
Hi Alex!
> Is Tsets a complete number of all various combinations of all parameters?
Yes, you are right: A tset is an interaction: For example, if we have Parameter 1 with values A and B and Parameter 2 with values C and D, all the (pairwise) tsets are: {A, C}, {A, D}, {B, C} and {B, D}. Notice that {A, B} and {C, D} are illegal and invalid tsets.
In the numbers you see (21424 / 21424 (0)), the first number is the number of tsets covered by the tests. The second number is the total number of legal and valid tsets. The third number is the difference between those numbers: The uncovered tsets. If the coverage is 100% the number of uncovered tsets is 0.
We have planned to add some explanation of those numbers in addition to adding other interesting metrics.
> How did you figure out that 293 tests covers all necessary situations?
It is basically a greedy set cover algorithm. For an overview, you can read this paper:
http://martinfjohansen.com/papers/Johansen2012.pdf
The Pairwiser algorithm is based on this algorithm with some significant advancements that makes it several orders of magnitudes faster.
> How do you evaluate for example that 16 testcases in this situation will cover 82% of functionality?
It is done using the algorithm and associated algorithms mentioned in the previous answer.
I did run the test generation task submitted on 20150629 12:24:53 (CET) on a larger server.
The result has 63 759 test cases and took 21 494 937 CPU ms to generate (approx 6 CPU hours). On a machine with 8 CPUs, this will take less than 1 hour and on a machine with 64 CPUs less than 10 minutes (probably).
Hi Alex!
Pairwiser can indeed handle your models, but it is a question of the size of the servers we are running on. Not many people run such large models (yet!) so the servers you ran on was not fast enough for a swift response to such a large model. (By the way, we will soon provide products which will run on larger servers. We will also put up a progress bar during test generation, so you can see the progress.)
Let me first see if I have found the correct model. Your model has, among other things, two parameters called “Include countries” and “Exclude countries”, each with a long list of countries?
This is an unusual way to set up a model for Pairwise testing. It will necessarily cause the number of tests to be very high. It will be 10 000 if not 100 000 tests.
Pairwise testing performs best with models with many parameters (even 10 000s) but with relatively few values each. (e.g. less than 10.) This will cause a few tests to cover a lot of interactions. With a few parameters and many values (such as in your model), each tests can only cover a few interactions, causing a large amount of tests and slow generation speeds.
I would be happy to look at you testing problem and assist you in setting it up! There are techniques that can be applied, such as equivalence classes, to make the testing problem perform well with pairwise testing. We can either discuss it here on the forums or just send me a mail on [email protected], and I will assist you in setting it up and making sure that you get your tests successfully generated, even if your models are very large.

AuthorPosts