Browser

Python, Selenium Grid and Docker

With Docker you can quickly and easily install, configure and use Selenium Grid. This tutorial shows the respective steps that you need as a software tester (or Developer). Instead of Python you can also use other languages, which are supported by Selenium​.

Preconditions

Preparation of files

# create new project
$ mkdir -p ~/Project/SeleniumTutorial && cd ~/Project/SeleniumTutorial

# create docker-compose.yml (version 1)
$ vim v1-docker-compose.yml

# or create docker-compose.yml (version 2)
$ vim v2-docker-compose.yml

# create python example.py
$ vim example.py

Note: You can opt for a version of docker-compose.yml!

Version: 1

---
selenium_hub:
  image: selenium/hub
  ports:
    - 4444:4444
node_1:
  image: selenium/node-chrome
  links:
    - selenium_hub:hub
node_2:
  image: selenium/node-firefox
  links:
    - selenium_hub:hub

Version: 2

---
version: '2'
services:
  selenium_hub:
    image: selenium/hub
    ports:
      - 4444:4444
  node_1:
    image: selenium/node-chrome
    depends_on:
      - selenium_hub
    environment:
      - HUB_PORT_4444_TCP_ADDR=selenium_hub
  node_2:
    image: selenium/node-firefox
    environment:
      - HUB_PORT_4444_TCP_ADDR=selenium_hub
    depends_on:
      - selenium_hub
import os
import datetime
import time
import unittest
from selenium import webdriver


class Example(unittest.TestCase):

    def setUp(self):

        self.driver = webdriver.Remote(
            command_executor='http://192.168.99.100:4444/wd/hub',
            desired_capabilities={
                'browserName': 'firefox',
                'javascriptEnabled': True
            }
        )

        self.driver.get('http://softwaretester.info/')

    def test_something(self):

        dt_format = '%Y%m%d_%H%M%S'
        cdt = datetime.datetime.fromtimestamp(time.time()).strftime(dt_format)
        current_location = os.getcwd()
        img_folder = current_location + '/images/'

        if not os.path.exists(img_folder):
            os.mkdir(img_folder)

        picture = img_folder + cdt + '.png'
        self.driver.save_screenshot(picture)

    def tearDown(self):

        self.driver.quit()


if __name__ == "__main__":

    unittest.main(verbosity=1)

Create environment

# create new VM
$ docker-machine create -d virtualbox Grid

# pointing shell
$ eval $(docker-machine env Grid)

# show status (optional)
$ docker-machine ls
...
NAME   ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
Grid   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.11.1 

# run docker-compose (Version: 1)
$ docker-compose -f v1-docker-compose.yml up -d

# run docker-compose (Version: 2)
$ docker-compose -f v2-docker-compose.yml up -d

# show status (Version: 1)
$ docker-compose -f v1-docker-compose.yml ps
...
             Name                         Command           State           Ports          
------------------------------------------------------------------------------------------
seleniumtutorial_node_1_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_node_2_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_selenium_hub_1   /opt/bin/entry_point.sh   Up      0.0.0.0:4444->4444/tcp

# show status (Version: 2)
$ docker-compose -f v2-docker-compose.yml ps
...
             Name                         Command           State           Ports          
------------------------------------------------------------------------------------------
seleniumtutorial_node_1_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_node_2_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_selenium_hub_1   /opt/bin/entry_point.sh   Up      0.0.0.0:4444->4444/tcp

Open Browser

Selenium Grid Console

Run Python script

# run python selenium script
$ python -B ~/Projects/Selenium/example.py

Note: Via browserName (example.py) you can choose the respective browser (firefox or chrome)!

Note: Via docker-compose scale you can add/remove node instances!

# create 2 instances (Version: 1)
$ docker-compose -f v1-docker-compose.yml scale node_1=2

# create 3 instances (Version: 2)
$ docker-compose -f v2-docker-compose.yml scale node_2=3

Headless browser

Some headless browser for test automation

  • Guillotine – A .NET headless browser, written in C#
  • PhantomJS – a headless WebKit scriptable with a JavaScript API
  • Zombie – Insanely fast, full-stack, headless browser testing using node.js
  • CasperJS – a navigation scripting & testing utility for PhantomJS and SlimerJS
  • HeadlessBrowser – headless browser, for testing the DOM on Node.js
  • SlimerJS – a scriptable browser for Web developers
  • trifleJS – a headless Internet Explorer browser using the .NET
  • Jabba-Webkit – a headless webkit browser for scraping AJAX
  • HtmlUnit – a GUI-Less browser for Java programs
  • Awesomium – HTML UI Engine for C++ and .NET
  • env.js – a pure JavaScript browser environment

HAR with Python WebDriver and BrowserMob Proxy

This time is shown how to automate performance test for web sites. Tools in usage are Python Selenium WebDriver and BrowserMob proxy. The results are HAR files which can be viewed in HAR Viewer.

Precondition

  • JAVA installed
  • Python Packages for selenium and browsermob-proxy
selenium
browsermob-proxy

Preparation

Download BrowserMob Proxy and check if proxy can started by command-line.

# change to 
$ cd browsermob-proxy-2.1.0-beta-1/bin/

# start proxy on port 9090
$ ./browsermob-proxy -port 9090

Create Python Class

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Python - BrowserMob - WebDriver"""
from browsermobproxy import Server
from selenium import webdriver
import json


class CreateHar(object):
    """create HTTP archive file"""

    def __init__(self, mob_path):
        """initial setup"""
        self.browser_mob = mob_path
        self.server = self.driver = self.proxy = None

    @staticmethod
    def __store_into_file(title, result):
        """store result"""
        har_file = open(title + '.har', 'w')
        har_file.write(str(result))
        har_file.close()

    def __start_server(self):
        """prepare and start server"""
        self.server = Server(self.browser_mob)
        self.server.start()
        self.proxy = self.server.create_proxy()

    def __start_driver(self):
        """prepare and start driver"""
        profile = webdriver.FirefoxProfile()
        profile.set_proxy(self.proxy.selenium_proxy())
        self.driver = webdriver.Firefox(firefox_profile=profile)

    def start_all(self):
        """start server and driver"""
        self.__start_server()
        self.__start_driver()

    def create_har(self, title, url):
        """start request and parse response"""
        self.proxy.new_har(title)
        self.driver.get(url)
        result = json.dumps(self.proxy.har, ensure_ascii=False)
        self.__store_into_file(title, result)

    def stop_all(self):
        """stop server and driver"""
        self.server.stop()
        self.driver.quit()


if __name__ == '__main__':
    path = "browsermob-proxy-2.1.0-beta-1/bin/browsermob-proxy"
    RUN = CreateHar(path)
    RUN.start_all()
    RUN.create_har('google', 'http://google.com')
    RUN.create_har('stackoverflow', 'http://stackoverflow.com')
    RUN.stop_all()

Note: The highlighted line must contain the path to BrowserMob Proxy!
Happy testing! If you use PhantomJS instead of Firefox, you can use this on Build server like Jenkins/Hudson and so on, too.

Test responsive webpages online

Of course it is better to test webpages directly on devices, but in case of costs not possible. A first good and cheap possibility is the use of online resources, next to browser Add-ons. Developer and software tester can use this services for a good impression. In most cases, the specifying of the URL and sizes are sufficient. This topic show some good free services. Which is the right one for you, you must decide for yourself.

Online Services

UI testing with DalekJS and PhantomJS on CentOS

With DalekJS you can automate your functional GUI tests very easy. This article describe how to prepare the test environment on CentOS.

Setup

First, the necessary packages are installed.

# install EPEL (Extra Packages for Enterprise Linux)
$ yum install -y epel-release

# check repositories
$ yum repolist

# update all
$ yum -y update

# install packages
$ yum install -y vim nodejs npm bzip2 freetype fontconfig

# check versions of nodejs and npm
$ node --version
$ npm --version

# change to tmp folder
$ cd /tmp/

# download PhantomJS and unzip
$ curl -O https://phantomjs.googlecode.com/files/phantomjs-1.9.2-linux-x86_64.tar.bz2
$ tar xvf phantomjs-1.9.2-linux-x86_64.tar.bz2

# copy binary, delete unneeded files and check version
$ cp phantomjs-1.9.2-linux-x86_64/bin/phantomjs /usr/local/bin/
$ rm -fr phantomjs-1.9.2-linux-x86_64
$ rm -f phantomjs-1.9.2-linux-x86_64.tar.bz2
$ phantomjs --version

# install dalek-cli (global) and check version
$ npm install dalek-cli -g
$ dalek --version

It is also possible to compile PhantomJS itself, but this takes a lot of time.

Prepare test project

Once all is installed without any issues you can start to create the test project.

# change to home folder
$ cd ~/

# create project folder and change into
$ mkdir DalekTest && cd DalekTest

# create package.json interactively
$ npm init

# install dalekjs on project
$ npm install dalekjs --save-dev

Create test case

Now it is time for the first Test Case. I have used the example from Dalek website.

# create test case file
$ touch test_title.js

# insert content
$ vim test_title.js
module.exports = {'Page title is correct': function (test) {
  test
    .open('http://google.com')
    .assert.title().is('Google', 'It has title')
    .done();
  }
};

Run test

By default DalekJS use PhantomJS as browser. For running the test case simple use dalek command and as argument the test case file (*.js).

# run test case
$ dalek test_title.js

Lynx text based web browser

Lynx can be used to check web site for accessibility, performance and SEO analysis.

Install Lynx

# install with yum
$ yum install -y lynx

# install with apt-get
$ apt-get install -y lynx

# install with Mac ports
$ sudo port install lynx

Use Lynx

# textual representation of the web page
$ lynx http://softwaretester.info

# print response header
$ lynx -head http://softwaretester.info
$ lynx -dump -head http://softwaretester.info

# show source code
$ lynx -source http://softwaretester.info

# grep links to STDOUT
$ lynx -dump http://softwaretester.info | grep "http:"

# count internal links
$ lynx -dump http://softwaretester.info | grep -o "http://softwaretester.info" | uniq | wc -l

# store external links into file
$ lynx -dump "http://softwaretester.info" | grep -o "http:.*" | grep -v "http://softwaretester.info*" > file.txt

Local HAR viewer on Mac OS X

There are several HAR file viewers online but sometimes you need the HAR viewer offline. It is very simple for Mac OS X user to get a local instance running.

Precondition

Preparation

The first step is generating HAR file.

# create HAR log file
$ phantomjs netsniff.js "http://google.com" > ~/Desktop/result.har

Now download the latest Harviewer.zip and unzip into the user “Sites” folder. Rename the folder and setting up the permissions.

# go into Downloads
$ cd ~/Downloads/

# create new folder
$ mkdir harviewer

# unzip into new folder
$ unzip harviewer-2.0-15.zip -d harviewer

# move folder into user sites
$ mv harviewer ~/Sites/

# go into sites and change access rights
$ cd ~/Sites/
$ chmod +x harviewer/

Result

Now open a browser and call URL like: “http://localhost/~<user>/harviewer/“. As last step drag the generated HAR file into the browser. You should see something like this:

HAR viewer result

Full webpage screenshot

For various reasons screenshots for webpages are needed. If automated test scripts fail, documentations must be created or in some other situations. With PhantomJS it is very easy to create these screenshots very fast by command-line. All what is needed a small JavaScript like this.

JavaScript

var phantom;
var console;
var system = require('system');
var fs = require('fs');
var page = require('webpage').create();

var Info = {
	isDate: function () {
		'use strict';
		var mydate = new Date().toDateString();
		return mydate;
	},
	isTime: function () {
		'use strict';
		var mytime = new Date().getTime();
		return mytime;
	}
};

var Target = {
	isLocation: function () {
		'use strict';
		var args = system.args,
		    mylocation = 'http://google.com';
		if (args.length > 1) {
			mylocation = system.args[1];
			console.log('[LOG] use argument location');
		} else {
			console.log('[LOG] use default location');
		}
		return mylocation;
	},
	isSaveFolder: function () {
		'use strict';
		var folder = 'log';
		if (!fs.exists(folder)) {
			console.log('[LOG] creat directory ' + folder);
			fs.makeDirectory(folder);
		}
		if (!fs.isWritable(folder)) {
			console.error('[LOG] ' + folder + ' is not writable!');
			phantom.exit(1);
		}
		return folder;
	}
};

page.open(Target.isLocation(), function (status) {
	'use strict';
	switch (status) {
	case 'success':
		console.log('[LOG] page open successfully' + Info.isDate());
		var folder = Target.isSaveFolder();
		page.render(folder + fs.separator + Info.isTime() + '.png');
		phantom.exit(0);
		break;
	case 'fail':
		console.error('[LOG] page not open successfully');
		phantom.exit(1);
		break;
	default:
		console.error('[LOG] fail to open with unknown status:' + status);
		phantom.exit(1);
		break;
	}
});

After save the script you can run it like:

# screenshot of softwaretester.info
$ phantomjs screenshot.js "http://softwaretester.info"