Best practices of visual tests recording

How Screenster does it

When you are recording a new test, Screenster saves a screenshot at each your step, creating a visual baseline for this test. After the baseline has been created, it is time for an optimization run. Screenster takes new screenshots during a playback run and checks for differences from the baseline. If nothing has changed on this page, no ignore regions will be applied. If a CSS element has changed, Screenster will ignore it during the optimization replay presuming this element is a dynamic one: since it has changed during this short period of time, then probably it keeps changing all the way so there is little sense in analyzing these changes after each run.

Specifying the borders of an automatically applied ignore region is not always a simple task. Several such elements can be situated too close to each other causing Screenster to merge their separate ignore regions into a big one. So what happens if a big number of dynamic elements have been detected on a single page? If they are bordering one another or change their places during each playback, Screenster is likely to merge all these ignore regions. As a result a big piece of the page or even the whole page can get completely ignored.

Analyze your web product before recording visual tests

Of course, the situation when there is a lot of dynamic content on a page, is not rare. Many websites today contain lots of ads, videos, flash content, fast-spinning slider carousels etc. If you attempt to record a Screenster test on such a page, you can be sure that a good portion of it will get auto-ignored. That is how Screenster’s logic works. In such a case you can delete the automatically applied ignore region and try to add several ones manually – sometimes this allows you to have a number of more comprehensive smaller ignored areas.

But generally automated visual testing as a concept is just not suitable for web pages with high amount of dynamic content. This should be considered when creating use cases for a web application.

What else to consider: clean-up!

Another typical problem of automated visual tests is dealing with constantly appearing differences where the design and functionality should not actually change between runs. In most cases this is caused by tests causing changes by themselves – through modifying, deleting or adding new content. That is why it is so important to record automated visual tests which do not break the integrity of a web application under testing. For example, let us create an auto-test which verifies that email sending function is working. It should contain the following steps:

1. Log in to your account
2. Click to create a new message
3. Fill out the required fields: recipient address, subject, body
4. Click to send the message
5. Go to the ‘Sent messages’ folder to verify that it was sent successfully

Now, if you follow these exact steps while recording an automated visual test, you will never get consistent results – on the contrary, this test will return failure in 100% of cases. The reason is that at the 5th step the page will look different after each test run because a new sent message item will keep getting added to the list. The solution? An automated test which makes changes to the application it runs against, requires clean-up to be done at the end.

Proper test steps for the above scenario are:

1. Log in
2. Create new message
3. Fill out the fields
4. Send
5. Check the ‘Sent’ page and
6. Delete the message you have just sent.

Clean-up job should always be done if your test case includes modification of web app content. Ensuring that your product is applicable for visual regression testing automation, and that your test scenarios are consistent, will help you getting best experience with Screenster!

But there’s more to learn so check out the following sections about other cool Screenster features or watch a video about automating visual regression testing of Gmail interface.



Please visit the Release History page to check what is new and what has been improved in your current Screenster version.