Using of Selenium WebDriver with py.test for test automation. Part #1

This article is the second part in the series of articles that cover Selenium and Unit Testing. All configuration stuff as well as usage of different approaches for unit testing was discovered in the first part.

Structure of the article is as follows:

• Infrastructure for test development and launching
• Special tool for communication with system under test through browser. In our case it is WebDriver.
• Special design patterns that were developed for such kind of tasks.

Let’s talk about test automation system. It consists of:

• The system or application that we would like to test and it is available for us through a browser like a web-based application.
• Special driver that allows us to send special commands to the application or receive information from the browser that were required to perform our tests. In our case it is Selenium
• Tests that we can develop or record using special tools.
• Framework for test launch. It may support functionality for launching selected or pre-defined group(s) of tests. Most known systems for Python are unittest, nosetest and py.test

So, Selenium WebDriver is a library that allows to develop tests that communicate with browser. For example, it can receive required data from browser or send special commands to the browser that can change its behavior. Selenium has special IDE that allows test recording and code generation. Also Selenium package includes Selenium Server component, it is required if we would like to launch our tests remotely. Selenium supports more known browsers like FireFox, Chrome, Internet Explorer, etc. For our article we will use FireFox, but let’s download driver for browser Google Chrome to demonstrate how tests work with both FireFox and Google Chrome. Selenium WebDriver has the capability to launch tests remotely, using Remote browser; it should work the same way for both FireFox and Chrome.

In our case workflow will have the following structure: Tests ↔ FireFox Driver ↔ FireFox ↔ Application that we would like to test.

To isolate our experiment from the original installation of the Python we will use virtual environment. Detailed information how to install it, configure it and use with Eclipse you may find in this article.

Since Selenium is needed for web-based application automation, the following list of functionality should be reviewed. It consists of actions needed for test automation development:

• Launch the web-browser
• Open the web-page
• Find the required elements (with delays and without)
• Collect information from elements on the web-page
• Perform the required actions with elements
• Quit the browser.

Sometimes switching between windows and frames is required. We will look on this functionality in the end.

Selenium home page for Python could be found on official website by the following link (http://selenium.googlecode.com/git/docs/api/py/index.html). Description of Selenium API is available via this link (http://selenium.googlecode.com/git/docs/api/py/api.html). That page is also available from the home page.

Let's quickly overview the list of commands and capabilities that Selenium framework provides to us. Since we work with a browser, Selenium has commands for browser launch and quitting it. Browser is an application so we are able to work with this window too. We can open, close it, and change size or position of the window. Selenium may also work with dialogue window for file uploading. Browser’s window contains web-page. Selenium provides us with the capability to open it, both back and next navigation, and executes JS code in context of the web-page. Each page contains elements. Selenium provides API to search elements on the page. Selenium supports emulation of mouse or keyboard actions as well as shows the properties of the web-page elements.

Launching of the browser.

Launching of the browser is executed by call of constructor for required driver

from selenium import webdriver

driver = webdriver.Firefox()

driver = webdriver.Chrome()

Each browser has its own class that represents the driver. Special constructor should be called to launch an instance of the required browser. Each browser has its own settings and list of settings is different for different browsers.

All browsers has one common parameter – capabilities (link on documentation is here (https://code.google.com/p/selenium/wiki/DesiredCapabilities)). Special instructions should be passed by using this way to change browser mode.

driver = webdriver.Firefox(capabilities={'native_events': False})

Command quit is used to stop the browser. It closes all opened windows and stop the driver. It clears all temporary files too.

driver.quit()

Command close closes current browser window, if it latest window then stop the browser. In some cases Selenium cannot identify that current window is the latest one, and as a result, it can be cause of situation when temporary files presents in the system after end of work.

driver.close()

Behavior for remote launching (with Selenium Grid) are differ from local launch. The quit function clear the session (session is moved to inactive state) and session is available for launching of new browser instance. The close function only stops the browser and do not clear the session, as a result the session will be available for next launch only after timeout.

Page opening and navigation

Function get provides capability to open web-page. This function accepts full path to the required page:

driver.get("https://www.yandex.com")

This command works synchronously. It waits till the moment when page loading process is finished. It is depends on browser.

Navigation by history is supported by the following list of commands:

driver.back()

driver.refresh()

driver.forward()

A hidden problem is related when a page has completed form and during back/refresh/forward navigation dialogue window appears with question to resubmit values from the form. Selenium does not support this feature.

Search elements on page

Selenium has two commands that can be used to find element on the page:

element = driver.find_element(by, locator)

elements = driver.find_elements(by, locator)

The first command looks for first element occurrence by special condition, the second command returns list of all elements that satisfy to special condition.

Different approaches for type of search exists:

• By.ID: the element ID should be unique across the web-page, so, the By.ID is a better and faster approach to find the element on the page.
• By.XPATH: Initially XPath was developed for XML language. Description of XPath capabilities can be found here (http://www.w3schools.com/xml/xml_xpath.asp). It is more general approaches that has more features than By.CSS_SELECTOR locator.
• By.LINK_TEXT: search by text of the link. This approach works only with link elements. Keep in mind that if application supports several languages, this approach may works incorrectly.
• By.PARTIAL_LINK_TEXT: the same like for By.LINK_TEXT
• By.NAME: this approach is used with input fields and buttons, because these elements usually has attribute name.
• By.TAG_NAME: search by name of the html tag.
• By.CLASS_NAME: search by class of the elements on the page.
• By.CSS_SELECTOR: locators that described in Cascading Style Sheets (CSS) standard (http://www.w3.org/Style/CSS/)

Selenium provides functions where locator's type included into function's name.

driver.find_element_by_id(<id to search>)

driver.find_elements_by_id(<id to search>)

driver.find_element_by_xpath(<xpath to search>)

driver.find_elements_by_xpath(<xpath to search>)

driver.find_element_by_link_text(<link text to search>)

driver.find_elements_by_link_text(<link text to search>)

driver.find_element_by_partial_link_text(<partial link text to search>)

driver.find_elements_by_partial_link_text(<partial link text to search>)

driver.find_element_by_name(<name to search>)

driver.find_elements_by_name(<name to search>)

driver.find_element_by_tag_name(<tag name to search>)

driver.find_elements_by_tag_name(<tag name to search>)

driver.find_element_by_class_name(<class name to search>)

driver.find_elements_by_class_name(<class name to search>)

driver.find_element_by_css_selector(<css selector to search>)

driver.find_elements_by_css_selector(<css selector to search>)

Selenium also supports search inside web-element by surrounding search requests in chain:

element = driver.find_element_id(<id>).

                 find_element_by_xpath(<xpath>).

                 find_element_by_name(<name>)

It is equals to:

element_with_id = driver.find_element_id(<id>)

element_with_xpath = element_with_id.find_element_by_xpath(<xpath>)

element = element_with_xpath.find_element_by_name(<name>)

If find_element function does not find anything, the NoSuchElementException will be raised, but the find_elements function returns empty list in this case. The exception or empty list can be returned not at the moment when function is called - implicit wait can be used to notify WebDriver to wait required time till check element's occurrence on web-page:

driver.implicitly_wait(<time to wait>)

Selenium uses explicit wait too. It is call of function time.sleep(<small time-out: ~0.5 sec>). Selenium encapsulates such behavior into class WebDriverWait:

from selenium.webdriver.support.wait import WebDriverWait

# initialization of explicit wait object

wait = WebDriverWait(<instance of WebDriver>, <time-out to wait>)

# using explicit wait object

element = wait.until(<condition for wait something>)

With the explicit wait functionality, we can wait for special conditions that is not possible to catch by implicit wait functionality. List of expected conditions available here (http://selenium.googlecode.com/git/docs/api/py/webdriver_support/selenium.webdriver.support.expected_conditions.html) and several good examples also available here (https://blog.mozilla.org/webqa/2012/07/12/how-to-webdriverwait/). Positive conditions or events should be checked to avoid tests slowness due to using of the explicit wait mechanism. For example, if web-page should have element a1 and shouldn't have element a2, then required condition should checks presence of element a2, and does not check absence of element a1 in source code of tests.

Differences between implicit wait and explicit wait are:

• Explicit waits works on client side, in Python source code, but implicit waits works on browser side
• Explicit waits can wait for big list of different conditions, but implicit waits waits for element appearance in DOM model
• Explicit waits should be written by the developer, but implicit waits work automatically
• Explicit waits can raise the TimeoutException, but implicit waits can raise NoSuchElementException
• Explicit waits generates a lot queries across the network, but implicit waits generates only one query that will be send to the browser.

Actions with the element

Simple actions:

• Click on links, buttons, radiobuttons and checkboxes. Select an item from the list. Clicks are emulated by command click.
• Enter text and attach file (enter special text into special input item). Text input emulated by command send_keys

Complicated actions:

• Combined keyboard actions (Ctrl + something, etc.) or navigation by arrows
• Homing or put mouse on top of element without click
• Drag and drop (with keyboard interaction)
• Right mouse buttons click

Now more details about two basic operations: click and send_keys. Both commands can be executed with any visible element and events will be generated during emulation, for example for the click command it will be mouse-down -> on-click -> mouse-up, and so on. The click command sends action at center of the element. The send_keys command adds text at the end of existed text in the element. The send_keys command can process key combinations, navigation arrows, and so on.

Operations with elements like select/toggle/check/uncheck can be emulated by series of click commands. But the select operation is a complicated one and its support has been added into WebDriver as a part of support package:

from selenium.webdriver.support.select import Select

dropdown = Select(element)

dropdown.select_by_index(1)

This class supports operations select_by_value and select_by_visible_text also.

For example if text should be entered at the beginning of the string, following list of commands is required:

from selenium.webdriver.common.keys import Keys

element.click()

element.send_keys(Keys.HOME)

element.send_keys(<some text>)

Both commands click and send_keys has two implementations for each browser: native and synthetic:

• Native events works on OS level, but synthetic works inside the browser.
• Native events is implemented on C/C++ programming language, but synthetic events is implemented using the JavaScript programming language.
• Native events emulates user behavior more clearly than synthetic.
• Native events requires focus on the browser, but synthetic not.
• Not all versions of the browser supports native events, but synthetic events works in all versions of the browser.

Lets look on complicated actions. It includes both click command and send_keys command as well as following list of complicated commands:

• move_to_element command
• click_and_hold command
• release command
• key_down command
• key_up command

Execution supported by ActionChains function. Following example demonstrates element drags and drops with the Ctrl button pressed.

Webdriver.ActionChains(driver).

    move_to_element(drag).

    key_down(Keys.CONTROL).

    click_and_hold().

    move_to_element(drop).

    release().

    key_up(Keys.CONTROL).

    perform()

Collect information about the element

• text. This property contains only visible text of the element. Invisible element have empty text.
• get_attribute. This function returns value of element's attribute. Sometimes this value can be changed, for example if attribute 'href' is requested. Selenium WebDriver returns absolute link (doesn't matter what kind of link was placed into the element initially).
• boolean attributes, these attributes has two kind of values: None or True. Selenium  returns none if attribute is not set, and True if attribute exists (it doesn't matter what value is stored in this attribute)
• Selenium returns value of the property with the same name like attribute name.
• The is_displayed method
• If required element locates before left border or above top border the is_displayed method returns true
• If required element partially covered by another element, the is_displayed method returns true
• If elements is transparent or its color are the same like background color, the is_displayed method returns true

Switch between content

The WebDriver allows switch to alert message, switch between frames and switch to another browser window. In details:

• Switch to alert:
• WebDriver has special method switch_to_alert, that return the alert if it exists. The NoAlertPresentException will be generated if no alert in the system, as well as UnhandledAlertException will be generated if no alert processing is implemented. Special function accept and dismiss of the alert object to close the alert message. Additional processing of alert window is covered by unexpectedAlertBehavior capability of the driver.
• Switch to frame. This capability is covered by function switch_to_frame
• Switch to window. Major point here is the following: the WebDriver doesn't support switching between tabs!
• List of all browser's windows can be get by function driver.window_handles()
• Handle of current window can be get by function driver.current_window_handle()
• Switch to required window by its handle can be performed by function driver.switch_to_window(<handle of required window>)
• Selenium doesn't switch back to previous window if current window was closed in program. Switch to another window (or previous window) should be done in the program too, otherwise WebDriver raises special exception.

Other browser window properties.

• get_window_size
• set_window_size
• maximize_window
• get_window_position
• set_window_position

These functions can be used for screenshot creating, avoid scrolling or for special checks with scrolling. Keep in mind, that Selenium WebDriver automatically performs auto-scroll operation if required element locates outside the window.

Implementation points

Common approach with development using Selenium is related to PageObject pattern usage. More detailed description on this pattern is here (https://code.google.com/p/selenium/wiki/PageObjects) This pattern makes test development process easier and allows to reduce amount of duplicated code into project. Benefits of PageObject pattern are:

• Clear difference between source code of tests and source code that depends on page structure
• Single repository for all web-page's services and operations

Example:

As example of usage Selenium Web-Driver as well as features and patterns from above, I will prepare simple automation for creating a mailbox on www.yahoo.com.

But it is a separate case for new post.

In any case, I've prepared initial architecture with same programming logic that could be found here.

Selenium + Python + unittest + nosetest + py.test. Start point.

In this article we are going to install and configure virtual environment in order to use it with such libraries like Selenium, nosetests, py.test and unittest.

So, to start development we need to install Python, after that Python package manager or pip. For Python we have an additional library which is useful for creating isolated Python environments: virtualenv.

Installation:

C:\Python27\Scripts>pip install -U virtualenv

Collecting virtualenv

Downloading virtualenv-13.1.2-py2.py3-none-any.whl (1.7MB)

100% |################################| 1.7MB 170kB/s

Installing collected packages: virtualenv

Successfully installed virtualenv-13.1.2

C:\Python27\Scripts>

After installing the virtual environment we should activate it and install Selenium WebDriver along with the frameworks that are going to be used for test development and execution. In my case it is unittest that includes into Python by default, nosetest and py.test.

Now, it is time to configure the environment.

First of all, we should create the required environment for our tutorial:

C:\Users\Mikhail\Documents\Development\virtual_environments>C:\Python27\Scripts\virtualenv.exe selenium_training

New python executable in selenium_training\Scripts\python.exe

Installing setuptools, pip, wheel...done.

C:\Users\Mikhail\Documents\Development\virtual_environments>

After that we should configure the new environment in Eclipse. Since we already have the project, we should go to the project's properties and click on the link 'Click here to configure an interpreter not listed' like in the picture below.

In the new window we should click the 'New' button, in the dialog you should enter a name for our environment and select Python's executable from our environment, like in the picture below:

After that, an additional window will be shown with the list of folders that should be included into Python's path:

Eclipse will process the new interpreter for several seconds

And after that we may choose this environment as the base for our project:

So, we created our environment and now it is the time to install nosetests, py.test and Selenium. Before the installation we should activate our virtual environment using activate command. Command source activate should be used in *nix based operation systems. After that we should deactivate it using deactivate command

C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training\Scripts>activate

(selenium_training) C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training\Scripts>pip install nose

Collecting nose

Downloading nose-1.3.7-py2-none-any.whl (154kB)

100% |################################| 155kB 435kB/s

Installing collected packages: nose

Successfully installed nose-1.3.7

(selenium_training) C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training\Scripts>pip install -U pytest

Collecting pytest

Downloading pytest-2.7.2-py2.py3-none-any.whl (127kB)

100% |################################| 131kB 440kB/s

Collecting py>=1.4.29 (from pytest)

Downloading py-1.4.30-py2.py3-none-any.whl (81kB)

100% |################################| 86kB 525kB/s

Collecting colorama (from pytest)

Downloading colorama-0.3.3.tar.gz

Building wheels for collected packages: colorama

Running setup.py bdist_wheel for colorama

Stored in directory: C:\Users\Mikhail\AppData\Local\pip\Cache\wheels\e3\24\8d\aec3db961cfbc8e939dc1843126548e7d479349f96659067e9

Successfully built colorama

Installing collected packages: py, colorama, pytest

Successfully installed colorama-0.3.3 py-1.4.30 pytest-2.7.2

(selenium_training) C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training\Scripts>pip install selenium

Collecting selenium

Downloading selenium-2.47.1-py2-none-any.whl (3.0MB)

100% |################################| 3.0MB 50kB/s

Installing collected packages: selenium

Successfully installed selenium-2.47.1

(selenium_training) C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training>deactivate

C:\Users\Mikhail\Documents\Development\virtual_environments\selenium_training>

The pip install command supports special requirements files. It is not a purpose of this article to describe how to do that – more information about this capability could be found here, as well as about the format of the requirement file – here

Selenium has integrated support only for FireFox. For other browsers we should download special executables. We expect that our tests will work in both Google Chrome and Mozilla FireFox. The special driver for Google Chrome is available here, the latest release is 2.19 (according to LATEST_RELEASE file). For Windows OS we should download a zip archive. This archive contains only one file chromedriver.exe. This file should be uploaded into directory from PATH environment variable.

The main purpose of this article is to demonstrate base usage of Selenium and several test frameworks on Python. A more detailed description will be discovered as part of several articles coming up next.

All the required modules and libraries have been installed and it is the time to make sure that all works fine.

Class for working with web-page

Both sites http://www.google.com and http://www.yandex.com have been used for test purposes of this article

I've prepared a simple class that has the following functionality:

• Start a browser's instance
• Open a page and return page's title
• Quit from the browser

Start a browser's instance is:

def start_browser(self, broswer_type='Firefox'):

    if re.match("firefox", broswer_type, re.I):

        self.driver = webdriver.Firefox()

    elif re.match("chrome", broswer_type, re.I):

        self.driver = webdriver.Chrome()

    else:

        self.driver = webdriver.Firefox()

We will support only two browsers Mozilla Firefox and Google Chrome with default browser Mozilla Firefox

Open a page and return page's title is:

def open_web_page(self, url):

    self.driver.get(url)

    return self.driver.title

An exit from the browser is covered by two separate functions. Selenium allows a user to open several windows in a browser. Due to this capability, Selenium has the quit() function that closes all windows in a browser and stops Selenium driver and the close() function that closes the current window of the browser, if the current window is the last window than the close() function works the same way as the quit() function. So, in our scenario, quit from the browser is:

def close_browser_window(self):

    self.driver.close()

    # Additional sleep for 5 seconds to make sure

    # that browser window has been closed properly.

    # If we remove that timeout, then call of function start_browser

    # after close_browser_window may do nothing, since instance exists

    # but closing

    sleep(3)

def stop_browser(self):

    # we should check if any instance of driver exists in the system

    if self.number_of_opened_windows() != 0:

        self.driver.quit()

Frameworks for writing and launching tests

Since a test may return both pass and failure results, each examples of the test framework usage will contain two types of scenarios:

• Scenario(s) that returns pass result
  
• Scenario(s) that returns failure
  

Simple tests with unittest

This my article provides enough information to get down to unittest module. Here I prefer to concentrate mainly on the key points of implementation:
Setup and teardown functions on class level are required to create a driver and start the browser instance, close all open windows and quit from browser respectively:

@classmethod

def setUpClass(cls):

    print "Unittest set-up class method"

    cls.page.start_browser()

@classmethod

def tearDownClass(cls):

    print "Unittest tear-down class method"

    cls.page.stop_browser()

Setup and teardown functions on method level are required to start the browser instance if it was closed due to the quit() function with the latest opened window of the browser and close current browser window respectively:

def setUp(self):

    print "Unittest set-up method"

    if self.page.number_of_opened_windows() == 0:

        print "Start new browser instance"

        self.page.start_browser()

def tearDown(self):

    print "Unittest tear-down method"

    self.page.close_browser_window()

Both correct and incorrect tests:

def testCorrectCheckTitle(self):

    actual_title = self.page.open_web_page(yandex_url)

    self.assertEqual(yandex_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(yandex_title, actual_title))

def testIncorrectCheckTitle(self):

    actual_title = self.page.open_web_page(google_url)

    self.assertEqual(yandex_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(yandex_title, actual_title))

Part of the output with failure and results:

======================================================================

FAIL: testIncorrectCheckTitle (__main__.UnitTestExample)

----------------------------------------------------------------------

Traceback (most recent call last):

File "C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training\unittest_example.py", line 39, in testIncorrectCheckTitle

self.assertEqual(yandex_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(yandex_title, actual_title))

AssertionError: Title is incorrect. Expected 'Yandex', Actual 'Google'

----------------------------------------------------------------------

Ran 2 tests in 43.165s

FAILED (failures=1)

I've highlighted the following information:

• Name of the failed test
• Total number of executed tests
• Amount of failed tests
• Assert message(s) for failed tests

Source code of the example is available here. Output log of the example is available here

Simple tests with nosetests

First of all, the documentation that is required to work with nose is available here. Pay your attention to this paragraph to understand the types of tests that could be launched by default within the nosetests framework: especially in this line: Any python source file, directory or package that matches the testMatch regular expression (by default: (?:^|[b_.-])[Tt]est) will be collected as a test (or source for collection of tests).
We can launch the unittest case and compare the results. A certain part of the log was removed so as to save space – please find the full log here. Please note that the reviewed case was launched from command line interface with activated virtual environment

(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>nosetests unittest_example.py
.F
======================================================================
FAIL: testIncorrectCheckTitle (mera.selenium_training.unittest_example.UnitTestExample)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training\unittest_example.py", line 39, in testIncorrectCheckTitle
self.assertEqual(yandex_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(yandex_title, actual_title))
AssertionError: Title is incorrect. Expected 'Yandex', Actual 'Google'
-------------------- >> begin captured stdout << ---------------------
Unittest set-up method
Start new browser instance
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
… Several lines were removed to save space …
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 2 tests in 44.355s
FAILED (failures=1)
(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>

We may notice that the format of the output differs from the unitest output format. The value of verbosity parameter by default equals to 1. It is the reason why we may notice .F at the beginning of the log. As we know from unittest framework . (dot) means that the test has been completed successfully and F (capital f) means failed result.
Lets talk about the nosetests itself. What kind of benefits we may achieve using this framework? The main advantage of nosetests usage is an extended version of fixtures. The nosetests framework has the following fixtures (depends on fixture's level):

• Test package: Nosetests provides the capability of grouping tests in packages. So, it allows package level setup and teardown function that should be declared in __init__.py module of the package. List of names for setup and teardown functions on package level is:
  
  • Setup functions: setup, setup_package, setUp, setUpPackage
    
  • Teardown functions: teardown, teardown_package, tearDown, tearDownPackage
    
• Test module: It is a module with name that satisfy to testMatch regular expression. List of names for setup and teardown functions on module level is:
  
  • Setup functions: setup, setup_module, setUp, setUpModule
    
  • Teardown functions: teardown, teardown_module, tearDown, tearDownModule
    
• Test classes: it could be a class in a module that matches testMatch regular expression or is a subclass of unittest.TestCase. At the level of classes both setup and teardown fixtures could have following names:
  
  • Setup functions: setup_class, setupClass, setUpClass, setupAll, setUpAll
    
  • Teardown functions: teardown_class, teardownClass, tearDownClass, teardownAll, tearDownAll
    
  • Like unittest.TestCase subclasses, other test classes can define setUp and tearDown methods that will be run before and after each test method.
    
• Test functions: set of functions in a test module that matches testMatch criteria. Major notice here is that test functions may define setup and teardown attributes, which will be run before and after the test function, respectively. For that purpose special decorator with_setup exists. This decorator is useful only for test functions and should not be used for test methods in unittest.TestCase subclasses or other test classes
  

Unittests module has a big list of assert functions. We may use it if our class is subclass of the unittest.TestCase class. But nosetests allows us to use the class that is not a subclass of unittest.TestCase class.
In this case we should use nose testing tools that are described here


Lets look at the simple implementation of tests according to the information above. As part of our tutorial we will look at test classes and test functions:
Our class has following structure:

class TestNoseTestExample():

    page = HomePage()

    # This method will be called before execution of tests from the class

    @classmethod

    def setup_class(cls):

        print "NoseTests set-up class method"

        cls.page.start_browser()

    # This method will be called before execution of each test

    def setUp(self):

        print "NoseTests set-up method"

        if self.page.number_of_opened_windows() == 0:

            print "Start new browser instance"

            self.page.start_browser()

    # This method will be called after execution of each test

    def tearDown(self):

        print "NoseTests tear-down method"

        self.page.close_browser_window()

    

    # This method will be called after execution of tests from the class

    @classmethod

    def teardown_class(cls):

        print "NoseTests tear-down class method"

        cls.page.stop_browser()

    def testCorrectCheckTitleYandex(self):

        expected_title = yandex_title

        actual_title = self.page.open_web_page(yandex_url)

        eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

    def testIncorrectCheckTitleYandex(self):

        expected_title = google_title

        actual_title = self.page.open_web_page(yandex_url)

        eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

    def testCorrectCheckTitleGoogle(self):

        expected_title = google_title

        actual_title = self.page.open_web_page(google_url)

        eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

    def testIncorrectCheckTitleGoogle(self):

        expected_title = yandex_title

        actual_title = self.page.open_web_page(google_url)

        eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

The differences between this implementation and the implementation using unittest.TestCase framework:

• Our class is not a subclass of unittest.TestCase. The nosetests supports such capability.
• Both setup_class and teardown_class methods is used to support setup and teardown operations on the class level
  
• Special functions eq_ from nosetest framework has the same role like assert of equal statement
  

The output has the same format as in the previous case. The only difference related to the name of the test method that has been caused:


(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>nosetests nosetest_example_class.py

..FF

======================================================================

FAIL: mera.selenium_training.nosetest_example_class.TestNoseTestExample.testIncorrectCheckTitleGoogle

----------------------------------------------------------------------

Traceback (most recent call last):

… Several lines were removed to save space …

----------------------------------------------------------------------

Ran 4 tests in 129.587s

FAILED (failures=2)

(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>

Let's demonstrate the usage of decorator @with_setup with functions. This capability is useful when we would like to add some flexibility to our tests. For example, we have several groups of functions and each group has its own setup and teardown. It can be solved by including each group in a separate class. But for list of functions we will achieve the same goal using the @with_setup decorator. For example, in our case let's set the site's URL in set-up level, not inside the test. I have two different URLs, so my setup and teardown methods have the following format:

# This method will be called before execution of special tests

def setup_yandex():

    global url, page

    print "NoseTests set-up method for Yandex"

    url = yandex_url

    if page.number_of_opened_windows() == 0:

        print "Start new browser instance"

        page.start_browser()

# This method will be called before execution of special tests

def setup_google():

    global url, page

    print "NoseTests set-up method for Google"

    url = google_url

    if page.number_of_opened_windows() == 0:

        print "Start new browser instance"

        page.start_browser()

# This method will be called after execution of each test

def teardown_any():

    global page

    print "NoseTests tear-down method"

    page.close_browser_window()

Tests has the following format (on example with one test):

@with_setup(setup=setup_google, teardown=teardown_any)

def testCorrectCheckTitleGoogle():

    global url, page

    expected_title = google_title

    actual_title = page.open_web_page(url)

    eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

Execution workflow starts with setup_module function, after that (for the test above) the setup_google function will be executed, after test execution the teardown_any function will be executed. The teardown_module function will be executed in the end, when the execution of the whole list of tests is finished.
The source code for the case with classes is available here, with functions – here.
The console output from the case with classes is available here, with functions – here.
The additional feature of nosetests is test generators support that described here. Pay your attention to the following point in the documentation:

• By default, the test name output for a generated test in verbose mode will be the name of the generator function or method, followed by the args passed to the yielded callable.
  
• If you want to show a different test name, set the description attribute of the yielded callable.
  
• Setup and teardown functions may be used with test generators. However, please note that setup and teardown attributes attached to the generator function will execute only once. To execute fixtures for each yielded test, attach the setup and teardown attributes to the function that is yielded, or yield a callable object instance with setup and teardown attributes.
  
• For generator methods, the setUp and tearDown methods of the class (if any) will be run before and after each generated test case. The setUp and tearDown methods do not run before the generator method itself, as this would cause setUp to run twice before the first test without an intervening tearDown.
  
• Please note that method generators are not supported in unittest.TestCase subclasses.
  

Lets look on following example:

# This method will be called before execution of all tests

def setup_module():

    print "NoseTests set-up module method for generators"

    page.start_browser()

# This method will be called before execution of special tests

def setup_any():

    global page

    print "NoseTests set-up method for generators"

    if page.number_of_opened_windows() == 0:

        print "Start new browser instance"

        page.start_browser()

# This method will be called after execution of each test

def teardown_any():

    global page

    print "NoseTests tear-down method for generators"

    page.close_browser_window()

# This method will be called after execution of tests from the class

def teardown_module():

    global page

    print "NoseTests tear-down module method for generators"

    page.stop_browser()

@with_setup(setup=setup_any, teardown=teardown_any)

def testCheckTitleFunction():

    for test_url, test_title in [[t_url, t_title] for t_url in [yandex_url, google_url] for t_title in [yandex_title, google_title]]:

    yield check_title, test_url, test_title

# Function that will be used in generator

def check_title(url, expected_title):

    global page

    actual_title = page.open_web_page(url)

    eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

# Class that will be used in generator (as callable object)

class CheckTitle():

    def __call__(self, url, expected_title):

        self.description = "Checking url: '{}' with title '{}'".format(url, expected_title)

        actual_title = page.open_web_page(url)

        eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))

@with_setup(setup=setup_any, teardown=teardown_any)

def testCheckTitleClass():

    for test_url, test_title in [[t_url, t_title] for t_url in [yandex_url, google_url] for t_title in [yandex_title, google_title]]:

    yield CheckTitle(), test_url, test_title

This case demonstrates the first three points from the description of the genarators above. This information is highlighted below. Please note that according to our implementation and the rule from the list above, first four tests (.FF.) uses an instance of the browser and second four tests (.FF.) uses another one

(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>nosetests nosetest_example_generators.py

.FF..FF.

======================================================================

FAIL: mera.selenium_training.nosetest_example_generators.testCheckTitleFunction('https://www.yandex.com/', 'Google')

… Several lines were removed to save space …

======================================================================

FAIL: mera.selenium_training.nosetest_example_generators.testCheckTitleFunction('http://www.google.com', 'Yandex')

… Several lines were removed to save space …

======================================================================

FAIL: Checking url: 'https://www.yandex.com/' with title 'Google'

… Several lines were removed to save space …

======================================================================

FAIL: Checking url: 'http://www.google.com' with title 'Yandex'

… Several lines were removed to save space …

Ran 8 tests in 60.867s

FAILED (failures=4)

(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>

Source code of the case with generators is located here.

Output from the execution of this code is located here.

My latest comment about nosetests is about the execution of the tests from Python scripts. We should use the following construction to perform the execution of all tests in a module:

if __name__ == "__main__":

    nose.runmodule()

Simple tests with py.test

The py.test is the framework with huge capabilities in terms of writing tests and test execution. I recommend that you review this framework in the on-linedocumentation but concentrate mainly on the following:

• Using py.test with functions and classes
• Parametrized test functions
• Marking test functions with attributes
• Skip and xfail: dealing with tests that can not succeed
• Continuously re-run failing tests

What is the correlation between py.test, nosetests and unittest? It is an standard question and you may find the answer (also given in the passage below) in py.test FAQ as well other useful information:

How does pytest relate to nose and unittest?

pytest and nose share basic philosophy when it comes to running and writing Python tests. In fact, you can run many tests written for nose with pytest. nose was originally created as a clone of pytest when pytest was in the 0.8 release cycle. Note that starting with pytest-2.0 support for running unittest test suites is majorly improved.

py.test framework, as well as nosetests framework, has its own test discoveryprocess. Lets repeat it here:

• collection starts from the initial command line arguments which may be directories, filenames or test ids.
• recurse into directories, unless they match norecursedirs pattern
  
• test_*.py or *_test.py files, imported by their test package name
  
• Test prefixed test classes (without an __init__ method). Source code is here and output is here
  
• test_ prefixed test functions or methods are test items. Source code is here and output is here
  

The py.test framework provides flexible mechanism of fixtures. The purpose is to provide a fixed base line upon which tests can be reliably and repeatedly executed. More information is available here

Highlights for the py.test framework:

• The py.test framework supports launching of tests that were developed with nosetest framework. There are several limitations to this case – you can find the details here. If we launch our case with nosetests functionality with functions, than output will be like that:(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>py.test.exe -s -v nosetest_example_functions.py
  ============================= test session starts =============================
  platform win32 -- Python 2.7.8 -- py-1.4.30 -- pytest-2.7.2 – c:\users\mikhail\documents\development\virtual_environments\selenium_training\scripts\python.exe
  rootdir: C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training, inifile:collected 4 items
  nosetest_example_functions.py::testCorrectCheckTitleYandex NoseTests set-up module method
  NoseTests set-up method for YandexPASSED
  NoseTests tear-down method
  nosetest_example_functions.py::testIncorrectCheckTitleYandex NoseTests set-up method for Yandex
  Start new browser instanceFAILED
  NoseTests tear-down method … Several lines has removed to save space … ================================== FAILURES ===================================
  ________________________ testIncorrectCheckTitleYandex ________________________
  @with_setup(setup=setup_yandex, teardown=teardown_any)
  def testIncorrectCheckTitleYandex():
  global url, page
  expected_title = google_title
  actual_title = page.open_web_page(url)
  > eq_(expected_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(expected_title, actual_title))
  
  nosetest_example_functions.py:68:
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
  a = 'Google', b = 'Yandex'
  msg = "Title is incorrect. Expected 'Google', Actual 'Yandex'"
  def eq_(a, b, msg=None):
  """Shorthand for 'assert a == b, "%r != %r" % (a, b)
  """
  if not a == b:
  > raise AssertionError(msg or "%r != %r" % (a, b))E AssertionError: Title is incorrect. Expected 'Google', Actual 'Yandex'
  ..\..\..\virtual_environments\selenium_training\lib\site-packages\nose\tools\trivial.py:29: AssertionError
  … Several lines has removed to save space …
  ==================== 2 failed, 2 passed in 124.12 seconds =====================(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>
  
  Full output is available here
  
• The py.test framework supports launching of tests that was developed using unittest.TestCase functionality. More information here. If we launch our unittest.TestCase example, than the output will look like that:(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>py.test.exe -v -s unittest_example.py
  ============================= test session starts =============================
  platform win32 -- Python 2.7.8 -- py-1.4.30 -- pytest-2.7.2 – c:\users\mikhail\documents\development\virtual_environments\selenium_training\scripts\python.exe
  rootdir: C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training, inifile:collected 2 items
  unittest_example.py::UnitTestExample::testCorrectCheckTitle Unittest set-up class method
  Unittest set-up method
  Unittest tear-down methodPASSED
  unittest_example.py::UnitTestExample::testIncorrectCheckTitle Unittest set-up method
  Start new browser instance
  Unittest tear-down methodFAILED
  Unittest tear-down class method
  
  ================================== FAILURES ===================================
  ___________________ UnitTestExample.testIncorrectCheckTitle ___________________
  
  self = <mera.selenium_training.unittest_example.UnitTestExample testMethod=testIncorrectCheckTitle>
  def testIncorrectCheckTitle(self):
  actual_title = self.page.open_web_page(google_url)
  > self.assertEqual(yandex_title, actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(yandex_title, actual_title))E AssertionError: Title is incorrect. Expected 'Yandex', Actual 'Google'
  unittest_example.py:39: AssertionError
  ===================== 1 failed, 1 passed in 41.72 seconds =====================(selenium_training) C:\Users\Mikhail\Documents\Development\sandbox_python\mera\selenium_training>
  Full output is available here.
  
• Several facts about assertion in tests:
  • Definition of the assert statement is: assert expression or assert expression custom_message
    
  • If the custom_message provided, then no assertion introspection takes places at all and the custom_message will be shown in the traceback.
    
  • Raising of exception can be checked with pytest.raises expression. More information about usage is here
    
  • The py.test framework provides smart comparison for a number of cases. Detailed information is here
    
  • User custom assertion could be add by implementation of the pytest_assertrepr_compare hook. As well as in previous cases, more details here
    
• Fixtures. As I mentioned above, fixtures in the py.test increase flexibility of tests' configuration significantly. In the documentation we may notice the following:
  • fixtures have explicit names and are activated by declaring their use form test functions, modules, classes or whole projects.
  • Fixtures are implemented in a modular manner, as each fixture name triggers a fixture function which can itself be fixtures.
    
  • Fixture management scales from simple unit to complex functional testing, allowing to parametrize fixtures and tests according to configuration and component options, or to re-use fixtures across class, module or whole test session scopes.

The number of fixture's capabilities is huge, I would like to concantrate only on top of the iceberg.

• Declaration of fixture can be done using a special decorator: @pytest.fixture
  
• Fixtures can be used in different scopes of a source code or a project. You should define the scope using the parameter with the name scope, default value is function. In my examples I'm using the scope module too. You may read about it here
  
• Fixtures are executed before the test method (like setup function). We could add a special finalizer method in fixture definition to make sure that the functionality of this method will be executed in the end (like teardown function):
   # This method will be called before execution of all tests@pytest.fixture(scope="module")def setup_module(request):
      print "Py.test set-up module method"
      page.start_browser()
      # This method will be called after execution of all tests      def fin(): 
          global page
          print "Py.test tear-down module method"
         page.stop_browser() 
     request.addfinalizer(fin)
  Fixtures without the finalizer method will not execute anything after test execution.
  This is described here

• Fixtures can be parametrized. This capability is described here. It allows to cover a large number of configurations with a small number of tests. Please look at my example here. This process of grouping fixtures is described here.
  
  • The py.test supports the parameters processing that needs to be skipped or produce the expected fail results – please read the description in official documentation. I've added my example that demonstrates the usage of xfail capability here. Output from the launch is here. This functionality is also supported in unittest.TestCase (Python 2.X or Python 3.X)
    
• The py.test supports both capabilities: parametrizing of fixtures and parametrizing of functions that is described here. You can find the the example and the output by following the links for fixtures source/output and functions source/output.
  I would like to provide several snippets of both source code with fixtures and source code with functions here:yandex_title = "Yandex"yandex_url = "https://www.yandex.com/"google_title = "Google"google_url = "http://www.google.com"
  For functions:@pytest.mark.parametrize("test_url,test_title",     [(yandex_url, yandex_title),
       (yandex_url, google_title),
       (google_url, google_title),
       pytest.mark.xfail((google_url, yandex_title))])@pytest.mark.usefixtures("setup_module", "setup_function")def test_CheckTitle(test_url,test_title):
      global page
      actual_title = page.open_web_page(test_url)
      assert test_title == actual_title, "Title is incorrect. Expected '{}', Actual '{}'".format(test_title, actual_title)
  Fixtures. The additional function id_fixture_function and the fixture's parameter ids are used for several purposes. The first purpose is to provide more description in the output (example of the output is here), and the second purpose is to filter the list of executed tests by special substring. If the test id contains this substring then the test will be executed (example of the output is here):# This function returns id for fixture valuedef id_fixture_function(fixture_value):        return "Url: '{}'. Title: '{}'".format(*fixture_value)
  @pytest.fixture(scope="function",    params = [[t_url, t_title] for t_url in [yandex_url, google_url] for t_title in [yandex_title, google_title]],
      ids = id_fixture_function)def setup(request):
      global url, title, page
      print "Py.test set-up method"    url = request.param[0]
      title = request.param[1]
      if page.number_of_opened_windows() == 0:
          print "Start new browser instance"       
          page.start_browser()
         # This method will be called after execution of each test           def fin():
              global page
              print "Py.test tear-down method"            page.close_browser_window()
          request.addfinalizer(fin)
  

If we need to execute the py.test inside Python script with tests, we should keep in mind that script's name must have one of the following formats test_*.py or *_test.py. Command below should be used for launching the py.test framework with this script. Parameters of the main function is a list of command line arguments that will be used if tests from the module are launched in command line:

if __name__ == "__main__":

    pytest.main(args=['-v', '-s'])