Automation

Headless browser

Some headless browser for test automation

  • Guillotine – A .NET headless browser, written in C#
  • PhantomJS – a headless WebKit scriptable with a JavaScript API
  • Zombie – Insanely fast, full-stack, headless browser testing using node.js
  • CasperJS – a navigation scripting & testing utility for PhantomJS and SlimerJS
  • HeadlessBrowser – headless browser, for testing the DOM on Node.js
  • SlimerJS – a scriptable browser for Web developers
  • trifleJS – a headless Internet Explorer browser using the .NET
  • Jabba-Webkit – a headless webkit browser for scraping AJAX
  • HtmlUnit – a GUI-Less browser for Java programs
  • Awesomium – HTML UI Engine for C++ and .NET
  • env.js – a pure JavaScript browser environment

Jenkins and Virtualenv

This guide is intended to show how you can use Jenkins/Hudson with Python virtualenv.

Precondition

Preparation

# install on Debian/Ubuntu/Mint
$ sudo aptitude update
$ sudo aptitude install python-pip python-virtualenv

# install on Mac OS
$ sudo easy_install pip
$ sudo pip install virtualenv

Example

Create (if necessary) a new “Freestyle Project” and configure as needed build-paramaters , VCS and etc. On section “Build” – “Execute Shell” insert following script.

# set variable
ExampleENV="${WORKSPACE}"/.ExampleENV

# delete folder and content if exists
if [ -d "$ExampleENV" ]; then
	rm -fr "$ExampleENV"
fi

# create new virtualenv
virtualenv --no-site-packages "$ExampleENV"

# activate virtualenv
. "$ExampleENV"/bin/activate

# CODE FOR VIRTUALENV...

Create PDF test-reports with Python

With only two python libraries and some line of codes, you are able to create very nice test-reports in PDF format. This tutorial should give you a hint. Feel free to improve with your requirements.

Preconditions

Example

matplotlib==1.4.3
reportlab==3.2.0

# -*- coding: utf-8 -*-

import time
import matplotlib.pyplot as plt

from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, Image
from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
from reportlab.lib.enums import TA_JUSTIFY
from reportlab.lib.pagesizes import letter
from reportlab.lib.units import inch


class CreateTestReport(object):
    """Example test report class"""

    def __init__(self, test_results, file_name):
        """Constructor method"""
        self.data = test_results
        self.file_name = file_name

        self.story = list()

        self.styles = getSampleStyleSheet()
        self.styles.add(ParagraphStyle(name='Justify', alignment=TA_JUSTIFY))

    def __create_overview(self):
        """Create overview graphic and pdf content"""
        requests = list()
        duration = list()

        for lists in self.data:
            requests.append(lists[0])
            duration.append(lists[1])

        plt.plot(duration)

        plt.xlabel('http requests')
        plt.ylabel('response time')
        plt.xticks(range(len(duration)), requests)

        plt.grid(True)
        plt.savefig('overview.png')

        text = '<font size=18>Overview</font>'
        self.story.append(Paragraph(text, self.styles["Heading2"]))
        picture = Image('overview.png', 6*inch, 4*inch, hAlign='LEFT')
        self.story.append(picture)

        text = '<font size=12>Lorem ipsum dolor sit amet, <br/>\
        consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt \
        ut labore et dolore magna aliquyam erat, sed diam voluptua.<br/>\
        At vero eos et accusam et justo duo dolores et ea rebum. <br/>\
        Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum \
        dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing \
        elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore \
        magna aliquyam erat, sed diam voluptua. <br/>\
        At vero eos et accusam et justo duo dolores et ea rebum. <br/>\
        Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum \
        dolor sit amet.</font>'
        self.story.append(Paragraph(text, self.styles["Normal"]))

        self.story.append(Spacer(1, 192))

    def __create_round_trips(self):
        """Create round trip graphics and pdf content"""

        for lists in self.data:
            text = '<font size=18>%s: %s HTTP/1.1</font>' % (lists[3],
                                                             lists[4])
            self.story.append(Paragraph(text, self.styles["Heading2"]))

            text = '<font size=12>Status Code: %s</font>' % lists[2]
            self.story.append(Paragraph(text, self.styles["Normal"]))
            self.story.append(Spacer(1, 12))

            file_name = 'request_' + str(lists[0]) + '.png'
            title = 'Request Circle (' + str(lists[1]) + ' sec.)'
            plt.figure(figsize=(4, 4))
            plt.title(title)
            values = lists[5]
            labels = ['Request', 'Server', 'Response']
            plt.pie(values, labels=labels)
            plt.savefig(file_name)

            picture = Image(file_name, 2.5*inch, 2.5*inch, hAlign='LEFT')
            self.story.append(picture)

            self.story.append(Spacer(1, 24))

    def create_pdf(self, report_title, author, base_url, margins):
        """Create PDF document"""

        formatted_time = time.ctime()
        pdf = SimpleDocTemplate(self.file_name,
                                pagesize=letter,
                                title=report_title,
                                author=author,
                                topMargin=margins[0],
                                leftMargin=margins[1],
                                rightMargin=margins[2],
                                bottomMargin=margins[3])

        # Add headline, created date/time and author
        text = '<font size=24>%s</font>' % report_title
        self.story.append(Paragraph(text, self.styles["Heading1"]))
        self.story.append(Spacer(1, 12))

        text = '<font size=10>%s<br/>by %s<br/>%s</font>' % (formatted_time,
                                                             author,
                                                             base_url)
        self.story.append(Paragraph(text, self.styles["Italic"]))
        self.story.append(Spacer(1, 24))

        # Add overview
        self.__create_overview()

        # Add results
        self.__create_round_trips()

        # Create and store report
        pdf.build(self.story)


if __name__ == '__main__':
    # some test result data
    data = ([1, 0.5, 200, 'GET', '/api/getUser', [40, 40, 20]],
            [2, 0.75, 200, 'POST', '/api/createUser', [50, 30, 20]],
            [3, 0.25, 404, 'PUT', '/api/updateUser', [50, 25, 25]],
            [4, 2, 301, 'GET', '/api/Logout', [25, 50, 25]],
            [5, 0.25, 200, 'POST', '/api/Login', [45, 35, 20]])

    # create report
    CREATE = CreateTestReport(data, 'TestReport.pdf')
    CREATE.create_pdf('Example Test Report',
                      'Lupin3000',
                      'http://example.com',
                      [20, 20, 20, 10])

You can call it simply by

# execute without compile
$ python -B ExampleReport.py

Create screenshot with Python Selenium Webdriver

The following example show how easy you could make screenshots with Python Selenium Webdriver.

Precondition

Example

# -*- coding: utf-8 -*-

import unittest
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException

from MyLibrary import MyLibrary


class Example(unittest.TestCase):
    """Example class for screen shot"""

    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(1)
        self.driver.get('http://softwaretester.info')

    def test_something(self):

        elem_id = 'is_not_there'

        try:
            elem = self.driver.find_element_by_id(elem_id)
            elem.click()
        except NoSuchElementException:
            MyLibrary.save_screenshot_picture(self.driver, elem_id)
            raise

    def tearDown(self):
        self.driver.close()

if __name__ == '__main__':
    unittest.main(verbosity=1)

The MyLibrary.py could be used on different places.

# -*- coding: utf-8 -*-

import os
import datetime
import time


class MyLibrary(object):
    """Simple screenshot class"""

    @staticmethod
    def get_date_time():
        """Return date_time"""
        dt_format = '%Y%m%d_%H%M%S'
        return datetime.datetime.fromtimestamp(time.time()).strftime(dt_format)

    @staticmethod
    def save_screenshot_picture(driver, file_name):
        """Make screenshot and save"""

        date_time = MyLibrary.get_date_time()
        current_location = os.getcwd()
        screenshot_folder = current_location + '/screenshot/'

        if not os.path.exists(screenshot_folder):
            os.mkdir(screenshot_folder)

        picture = screenshot_folder + file_name + ' ' + date_time + '.png'
        driver.save_screenshot(picture)

After running the test you should see the folder “/screenshot” with the picture.

JUnit report with Python Webdriver

To create JUnit reports, you just need the python library xmlrunner. This is needed for the integration with build server like Jenkins.

Installation

# example installation with pip
$ sudo pip install xmlrunner

Usage

Just replace for example:

unittest.TextTestRunner(verbosity=2).run(test_suite)

with

from xmlrunner import xmlrunner

""" code for test suite ... """

xmlrunner.XMLTestRunner(verbosity=2, output='reports').run(test_suite)

Jenkins

On Jenkins add a new “post-build” action and select the Publish JUnit test result report. Add now the string “reports/*.xml” into the “Test report XMLs” field.

DDT with Python Selenium

DDT (Data-driven Testing) with Python Selenium Webdriver is very easy! DDT becomes very useful if you have test cases that contains the same test steps. All values could outsourced into files or databases. This tutorial use CSV files.

Precondition

  • Python installed
  • selenium and ddt library installed

Example

The folder structure for this tutorial looks like:

├── data
│   └── scenario_a.csv
├── library
│   ├── GetData.py
│   └── __init__.py
├── scenarios
│   ├── __init__.py
│   └── scenario_a.py
└── testsuite.py

Into folder “data” we store the csv files. The packages “library” include a function to read the specific csv files and the package “scenarios” include the test cases. The test suite is on root folder.

#!/usr/bin/env python
# -*- coding: utf8 -*-

import unittest

from scenarios.scenario_a import TestScenarioA


# load test cases
scenario_a = unittest.TestLoader().loadTestsFromTestCase(TestScenarioA)

# create test suite
test_suite = unittest.TestSuite([scenario_a])

# execute test suite
unittest.TextTestRunner(verbosity=2).run(test_suite)

Into the “testsuite.py” we add all test cases provided by scenario package.

data folder

target_url,elem_name,search_value
http://softwaretester.info,s,python
http://softwaretester.info,s,selenium
http://softwaretester.info,s,webdriver
http://softwaretester.info,s,automation

The CSV stores the test data that we supplied to the @data decorator of test case.

library package

#!/usr/bin/env python
# -*- coding: utf8 -*-

__author__ = 'lupin'
#!/usr/bin/env python
# -*- coding: utf8 -*-

import csv


def get_csv_data(csv_path):
    """
    read test data from csv and return as list

    @type csv_path: string
    @param csv_path: some csv path string
    @return list
    """
    rows = []
    csv_data = open(str(csv_path), "rb")
    content = csv.reader(csv_data)

    # skip header line
    next(content, None)

    # add rows to list
    for row in content:
        rows.append(row)

    return rows

Just for read the csv and return the values as a list.

scenarios package

#!/usr/bin/env python
# -*- coding: utf8 -*-

__author__ = 'lupin'
#!/usr/bin/env python
# -*- coding: utf8 -*-

import unittest

from selenium import webdriver
from ddt import ddt, data, unpack

from library.GetData import get_csv_data


@ddt
class TestScenarioA(unittest.TestCase):
    """ inheriting the TestCase class"""

    @classmethod
    def setUpClass(cls):
        """test preparation"""
        cls.driver = webdriver.Firefox()
        cls.driver.implicitly_wait(3)
        cls.driver.set_window_size(450, 500)

    @data(*get_csv_data('./data/scenario_a.csv'))
    @unpack
    def test_search(self, target_url, elem_name, search_value):
        """test case for scenario a"""
        driver = self.driver
        driver.get(target_url)

        btn_elem = driver.find_element_by_id('search-toggle')
        btn_elem.click()

        input_elem = driver.find_element_by_name(elem_name)
        input_elem.clear()
        input_elem.send_keys(search_value)
        input_elem.submit()

    @classmethod
    def tearDownClass(cls):
        """clean up"""
        cls.driver.close()

Test case with @ddt (for classes), @data and @unpack (for methods) decorators.

  • @data take the arguments from csv file
  • @unpack unpacks tuples or lists into multiple arguments

The test_search() method accepts the arguments, which will be mapped to the tuple values by ddt.

Run

$ python -B testsuite.py

Python, Selenium and PhantomJS – ignore certificate errors

Background

You are done with your work and push all into Git. The Build-Server starts his work and all test scripts are failing. Short look and it is clear – certificate errors. The next example shows, how to ignore certificate errors on PhantomJS.

@classmethod
def setUpClass(cls):
    """starting point for test cases"""

    cls.driver = webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true'])
    cls.driver.implicitly_wait(30)
    cls.driver.set_window_size(1024, 768)
    cls.driver.get(cls.url)

Now it should work….

Startup with Jasmine

Jasmine is a free JavaScript testing framework for BDD (Behavior Driven Development). In my opinion Unit-test should be written by developers, but tester you need also to know! This tutorial give you a hint how to use it.

Preparation

Download latest standalone version directly from GitHub and unzip. Now you have some files and folders. The file “SpecRunner.html” and “spec”, “src” folders are important for us. Some examples are allready  included. If you run the “SpecRunner.html” in your browser, you can see the first example results. A look inside this examples help you understand Jasmine!

Short Introduction

In the “src” directory, are the things to be tested. The “spec” directory has the tests. The “SpecRunner.html” link to the all files. The comments describe what you should do. Whenever you want to run the tests, you simply need to load or reload.

Example

Delete all files into “src” and “spec” folders. Create on “src” folder a new file.

function say_hello(name) {
    return "hello from " + name;
}
function simple_calc(value_a, value_b) {
    return value_a + value_b;
}
function return_bool(value_a) {
    return isNaN(value_a);
}
var timo = {human: "male"};
var john = {human: "male"};

And on “spec” folder this file.

describe("my first test suite", function() {
    it("test for correct string", function() {
        expect(say_hello("Lisa")).toContain("hello from Lisa");
    });
    it("test for correct value", function() {
        expect(simple_calc(5, 6)).toEqual(11);
    });
    it("test for not a number", function() {
        expect(return_bool("Hello")).toBeTruthy();
    });
    it("test for same object", function() {
        expect(timo).toBe(john);
    });
});

Now edit the “SpecRunner.html”. Just add the paths for JS files.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>Jasmine Spec Runner v2.3.4</title>
  <link rel="shortcut icon" type="image/png" href="lib/jasmine-2.3.4/jasmine_favicon.png">
  <link rel="stylesheet" href="lib/jasmine-2.3.4/jasmine.css">
  <script src="lib/jasmine-2.3.4/jasmine.js"></script>
  <script src="lib/jasmine-2.3.4/jasmine-html.js"></script>
  <script src="lib/jasmine-2.3.4/boot.js"></script>
  <!-- include source files here... -->
  <script src="src/example.js"></script>
  <!-- include spec files here... -->
  <script src="spec/example.spec.js"></script>
</head>
<body></body>
</html>

That is it! 😉 Now your results shows 4 Specs with 1 failure.