Wednesday, February 17, 2016

Assertions are bad

Yup, I said it.  We're all thinking it.  We've all had problems with them, but no one has come out and said it.  

Don't use assertions with selenium-webdriver.  Use the webdriver api's wait methods instead.   Many people struggle with unstable tests, and this is one of the primary culprits.  

The reason is simple.  Modern web applications use a large amount of client-side javascript to render a web page.  This is to reduce the amount of HTTP requests that need to be made.  When a page is loaded javascript is usually the last thing to load, and then it reprocesses the page, rendering the DOM, and then modifying HTML, such as displaying an element or setting its text.  The user can even start interacting with the website, entering text, hiding or displaying modals, all without reloading the page. 

This means that there some time after which an element is findable by webdriver, but not in its final state.  During this timespan, if an assert is done, it will be incorrect, as the element isn't actually ready to be asserted.  

Wait methods available by the webdriver api are essentially assertions that have will try multiple times before failing the test.  Even a 5 second timespan can provide enough of a cushion that test stability can be improved drastically.  

The webdriver API's wait methods are available here and can be used for any condition.  If needed, they can be simplified and included as helper methods.  

Here I created a custom wait function that I can easily call from my code.  Now this only verifies the text, but it could easily be modified to be used for any other condition in the until() function.  

def wait_for_text(value, timeout=$element_timeout)
  wait("Element #{self} text was not correct after #{timeout} seconds.  Expected '#{value}' Got '#{element.text}'",timeout).until {
    element.text.include? value.to_s
  }
  $logger.info "Verified #{self} text contains '#{value}'"  selfend