Different approaches to dump and load objects using Python. Perfomance analysis.

The previous post gave you an example demonstrating the usage of json format to store an object's value. 

We have several ways to store/load (or serialize and deserialize) data that we collect during the launch of our application. 

At the moment we have several ways how to do that. First of all we should think about what we will do with that data? Will we load it back inside our application? Will we send it to another application (that could be written using another programming language, not especially Python)? Should we store data in the format that could be understood by a human, so that the stored data had a readable format?

What approaches might we have if we used Python?

1. JSON (http://json.org/)
2. XML (http://www.w3.org/XML/) 
3. YAML (http://yaml.org/ and RFC is here)
4. Pickle (Python 2 Docs, Python 3 Docs)

JSON, XML and YAML can be used to store information in human-readable format, and this data format can be uploaded using all modern programming languages. Pickle is a Python-specific module. An advantage of this module is that we can dump more complicated objects than standard data structures and types without any issues and special methods inside our classes that will convert instances of our classes into the format that is applicable to be dumped into XML, JSON or YAML formats.
So, it is the time to look at each bullet in a bit more detail.

First of all we should keep in mind that for serialization and deserialization we should have simple approach that allows us to store and load our data (simple variables, collections, objects) without any problems. We will have a look at implementations of modules with both dump and load functions.

JSON 

This format is supported by Python standard module (module description for Python 3.X). As you may see from documentation, the functionality provided in this module is pretty simple. It allows to serialize and deserialize an object into a string or a stream. The list of the possible types that can be encoded/decoded with this value includes: Several collection types: dict, list and tuple, String types like string and unicode string, Numeric types like int, long and float, Boolean values such as True and False, and of course None.
This library provides both dump function and load function. So, JSON is chosen for our experiment.

XML

XML support in Python is implemented in several modules. Full information can be found here. As I haven't found any applicable modules with the required functionality, I suggest that for our experiment we try to write our own module for serialization or deserialization, based on Expat module. But it is not the purpose of this article, so we will not use XML for this experiment.

YAML

Module PyYAML can be used for working with this markup language. It provides simple functions to dump and load information. Correlation with JSON is available here and with XML - here. This module has both dump and load functions, so we will use it for our experiment.

Pickle

Pickle is a standard module that allows to use serialization and deserialization processes for an object. It converts objects hierarchy in byte stream aka 'Pickling' operation and restore the objects hierarchy from byte stream aka 'Unpickling' process. This format is compared with JSON in the following article for Python 3.X
Since Pickle is a standard module, it also contains both dump function and load function, and we will use this module as part of our experiment.

Python unittest. A bit more practice after theory. Basics and a bit more than basics.

The main purpose of this article is just to share my experience on using unittest module in Python. 

I chose version 2.7.10 for this case, but I think we may expect the same behavior with Python 3.X

Documentation about unittest module available via following link: https://docs.python.org/2/library/unittest.html, the site interface with documentation is easy and you can simply switch to the same topic, but another version of Python.

In this article I will demonstrate my source code snippets with links for GitHub (where you can find full version)

Initially I've prepared a special class with functionality that we will be using to demonstrate unittest usage.

The class can be used for the following:

1. To generate a dictionary with random keys and values
2. To store generated dictionary like a JSON object into a file (by using standard library in Python)
3. To upload from the file the JSON object back to the dictionary

Latest version of the class could be found here (GitHub)

Now let's start with preparation of simple unit tests lists. You can find basic information about the unit test on Wikipedia. It provides enough knowledge to start working with that.

Let's switch to practice. 

Technical task. Test automation for web-page.

Recently I was asked about preparing of test-automation for the page http://hh.ru/price.
Major agreement: Java and Selenium.
So I would like to share several steps and ideas how I tried to solved required challenge.
I've created the Maven project to avoid problems with dependency for Selenium:

Adding Selenium dependency

Choose version of Selenium

To solve this task I preferred to show several technics, because adding tests with good architecture should not be difficult thing.

Serenity + JUnit for automation

Several days ago I was asked to complete one technical challenge task.

Task description is to prepare automation for set of settings for e-mail processing.

In details you should automate processing of the following:

One criteria is to use Serenity and JUnit

In this article I would not like to go deeper into tests. I would like to share several steps how I configure my project in Eclipse

Parametirized test. JUnit. JUnitParams.

I've faced with one very interesting opportunity during experiments with unit testing in Java.
Many thanks to codeabbey for a huge list of interesting tasks that allows me to extend my knowledge about unit testing.
First of all JUnit documentation has a lot of examples how to create parametirized tests. This page contains simple examples that very easy to understand.
But, current implementation of parametirized tests has one disadvantage.
You cannot use different set of parameters for different tests.
For example you implement some specific algorithms for sorting of array and searching for a value inside of array. You would not like separate unit tests to different classes and you want to have several input data sets for both sorting and searching. Or you would like to test behaviour of sort algorithms on Numeric data types and String data types separately. With current realization in JUnit you have no possibility to setup two or more different set of parameters. In other words, you cannot specify set of parameters for each unit test.
After searching in Google, I've found one interesting implementation of such possibility: using JUnitParams extension.
Following several examples can show how to use and how to manipulate with such tests:

1. Official WiKi page on github.com
2. Example from old project page code.google.com or new project page on github.com

Project has pom.xml file and can be build from master by Maven. Some corrections still required.

To make successful build for my unit testing I did the following:

1. Comment following code in pom.xml, to avoid installation gpg on my computer:
2. Change build version since latest version from master required java-1.8

If you will follow all steps from examples, than all will works fine.
It was very useful for me to find such package and use it in my unit tests.

Using of the WebDriver (Selenium). Some additional techniques

Before using web-driver you need to configure it. Please, make sure that all that you did corresponds to the following list of links:

1. Using Web Driver
2. FireFox Driver
3. Internet Explorer Driver

It was found out (I suppose no one would need any additional checks) that controls in web-application can have different location in hierarchy tree in different browsers. It means that some checks in your code need depend on a type of a web-driver. In my case, I actually did not find differences, so I did not write any checks on that.

What I've found useful is ExpectedConditions. A good example for using such capability can be found here and here.

How I automate testing of one web-application

I would like to share my experience with tests’ integration and preparation for a web-application, using Eclipse, Selenium and testNG.

A few words about the web-application that I need to automate. This is a 7-screen application. The screens have all kind of controls such as our custom control, edit boxes, labels, buttons, radio buttons, combo boxes, menu control and so on. I cannot provide any hard copies of screens or anything else about this tool because this information is confidential.

No doubt, you may find a lot of information on how to do it on the Internet. But I personally have failed to find a link that would contain all necessary information on one page, that’s why I would like to share my own experience.

Let me introduce the task. I have a rich web application developed with GWT. I have a specification, a list of test cases that I've prepared during the previous month and a requirement for additional automatization. I suggest that we have the same environment as our developers have. So I used Eclipse as IDE for development. After that I was trying out different test automatization. Based on my previous experience, the list was as follows:

1.  UIAutomation - Microsoft Visual Studio is required for development (no such IDE here). When I've looked through on the components tree of the application, I found out a huge hierarchy without any chance to identify required web-control.
2. AutoIT - Did not 'see' any controls on the web-application
3. WatIN - Microsoft Visual Studio is required for development (no such IDE here).

The first search sends me to Selenium project, so I read additional forums and articles, and decided to use it. 

The next step is test framework selection. Selenium gives an opportunity to generate the source code for Java and JUnit or testNG. According to this article, I decided to use testNG.  

First of all I tried to use Selenium for automatization, just to check if all significant cases can be covered by the Selenium IDE.

It took me a few days to make sure that it is possible to automate the main functionality of the web-application. 

However, I faced some troubles during my investigation.

1.   The name of test case will be converted to class name in Java during the test launch, so in the first place I suggest to avoid test names which contain special symbols like: 'Wheel: Speed 100 km/h', 'Wheel: Speed 200 km/h'

2.   Our application has a complex logic with many events, so I frequently needed to call operations with the “pause” command after it. First, I decided to use '*AndWait' versions of the required commands but it was not applicable for me according to this article (in details: Many Actions can be called with the “AndWait” suffix, e.g. “clickAndWait”. This suffix tells Selenium that the action will cause the browser to make a call to the server, and that Selenium should wait for a new page to load.)

3.   I had a lot of troubles with our custom control, because the 'sendKeys' function call was required to enter the text correctly. Function 'type' does not allow sanding the entered value correctly.

I probably should have been aware about these troubles but I was a beginner in Selenium.

Moreover, you can find information about how to create your first Selenium script by following this link

After that I decided to convert the existing Selenium tests from 'html' to 'java' and configure environment to work with these tests like from IDE.

If you want to learn how to configure a project in Eclipse you may follow this link (or this one and this one ). I used it for my project, but I had to make some changes at the last stage.

I decided to leave the functions generated by Selenium IDE as they are, I added them as separate functions, not as part of the main function. So, my main function looks like:

@Test

public void main() {

    try {

        ConvertedTestFunction();

    }

    catch(Exception e){

        System.out.print("Exception occurs: " + e.getMessage());

        Assert.fail(e.getMessage());

    }

}

'Assert' prefix was  added after I had converted my project to TestNG. (step #8 from link above)

After that I launched Selenium server using this command java -jar selenium-server-standalone-2.39.0.jar like has it was described in section 'Running Standalone Selenium Server for use with RemoteDrivers' of this page.  Afterwards I chose the menu item 'Run As -> TestNG Suite' in menu for testing.xml file (more information can be found in section 'Launching your tests in Eclipse' on this page)

Now all works fine!

Only one case in under investigation. function 'sendKeys' does not exist in realization on java.

Workaround in my case was in using both functions 'type("field", "text to type")' and 'typeKeys("field", "\\13")

I created the required ticket about this issue on official site.

As you can see using of Selenium RC is depricated. All test code should be updated from selenium to webdriver.