by functionaltesting » Wed Jun 04, 2008 10:15 am
A possible solution to a testing a web application with apparent
high-combinatoric combinations of input may be to use a continual
testing process with randomly chosen values for each input.
Of course, for a very large permutation count you will never likely
try every combination. But over time, as the total number of runs
increases, you have been able to accumulate enough successful runs
to generate some confidence in the integrity of your application.
Here is how this could work in practice.
* Create your test starting with an eValid recording of your web
application behavior.
* Parameterize that script with variables whose values are to be
assigned at the start of each test.
* Use eVgenerate (an eValid suite component) to create the values
for each instance of playback using the "random selection" option.
* Make sure errors (if any) are recorded systematically along with
the actual script that was used to cause them.
* Use the eValid batch (command line) interface and the Windows
scheduler to launch your test at regular intervals.
If you run one such test per minute, after a week you have about
10,000 samples and some number, N, of tests that fail.
This process does two things:
* If N continues to be zero, then you have growing confidence in the
quality of you web application.
* For every one of the N failures you have some evidence of a
problem, and evidence of something that needs to be fixed.
(This outline is an approximation of the "mutation testing" idea
from ancient testing history, to which a nod is given with respect.)
eValid Tech Support Team
