Image Validations Using Golem

Image Validations Using Golem

Published: May 27, 2014 by Brian Kitchener

WebDriver is an amazing tool, and it has the broadest market support of any open source tool on the market. However, many commercial tools offer something that WebDriver does not: the ability to perform a visual image comparison.

Image comparisons are useful to identify visual defects, and to determine CSS problems, rendering issues, and layout defects. However, they haven’t worked too well historically, as the slightest change in the UI would cause the validation to break. In addition, full-page screenshots were typically the only images supported, and even the slightest change in layout (such as running the test on a different computer) would make the validation fail.

To this end, the Golem framework (https://github.com/ProtoTest/ProtoTest.Golem) includes code to easily perform image-based validations on individual UI elements. This allows an individual button or panel to have its UI validated. In addition, it uses a configurable “fuzzy” image comparison algorithm that will adjust for different sized images, or slight rendering differences between runs.

So, an image that is recorded at one resolution in Firefox can be compared against an image that was recorded at a different resolution in Chrome. The comparison can be adjusted with two separate parameters to allow fine-tuning for each element.

In the following example, the compared source images must be built and stored by Golem. However, the underlying methods could be used with any two images.

Image validations are currently supported only for elements. It uses the name of the element, passed in during instantiation, to save the file to the hard drive. During test execution, if no image is found on the disk the current image is saved. This means that we must first execute our test against a “good” site, to have it capture and store the images. Every subsequent run will use those images as a comparison. If a source image needs to be updated, simply delete it off the hard drive and a new image will be saved the next time the test is run.

To perform an image comparison, we simply define our element and verify the image:

[Test]
public void TestImage()
{
driver.Navigate().GoToUrl("http://www.google.com");
Element searchButton = new Element("SearchButton", By.Name("btnK"));
searchButton.Verify().Image();
}

After we execute the test the first time, the image is stored to the hard drive in: ProjectDirectory\ElementImages\SearchButton.bmp

Subsequent runs will compare against this stored image. If we adjust our browser to run in chrome, the validation will pass, even though the size and font are slightly different between browsers. Now in the report we see:

To see what a test failure looks like, we will first edit our image and mark it:

Now when we execute the test, the report will show an error, and the difference between the images:

There you have it, a simple clean way to compare the visual look of UI elements. The Golem framework contains an example test using image comparisons here:https://github.com/ProtoTest/ProtoTest.Golem/blob/master/ProtoTest.Golem/Tests/TestImageComparison.cs

The GoogleHomePage.cs contains the actual call to verify the images:https://github.com/ProtoTest/ProtoTest.Golem/blob/master/ProtoTest.Golem/Tests/PageObjects/Google/GoogleHomePage.cs