forum.e-valid.com

How hard is to make a test "invulnerable" to unimportant changes, but still detect important changes?

Very good question.

The answer is easy enough: You have to decide "how invulnerable" you want the test to be.

There's this old-school idea, the "perfect test": it never fails, never reveals a fault, always PASSes. Perfect! But entirely useless!

On the other hand there is an equally "perfect test" which never PASSes, always finds something wrong with the object under test. Perfect! And equally useless.

The art is in deciding what is to be detected as a FAILure and what is not. In eValid tests of web applications, what you choose the validate (using the GUI to construct the commands or adding the validation commands into the script later) defines what you believe to be the minimal criteria for a PASS. If any of the validation tests FAIL, the test FAILS.

If you want your test to igore the positions of things, i.e. how the page is organized, then you will have to edit your recorded script to include what we call "structural testing passages," which pivot the test on some fixed point in the page that you KNOW won't be changing. This usually amounts to identifying a table row, or some image, or some other page feature using an IndexFindElement or IndexFindElementEX command and pivoting the test around that location in the page.

Such tests, when combined with rigorous collections of validation steps, go along way toward creating a truly "invulnerable test" that successfully validates what needs to be validated, but does not give you a FAIL when the changes are (as you've determined) due to objects moving around on the page.

eValid Technology Team