pylint and lxml

If you use Python virtualenv, pylint and lxml together, you may see error messages in pylint test results. It`s because only trusted C extension resources (the standard library) should be used. Here is an opportunity to improve the pylint test results.

Generate a .pylintrc file

# generate .pylintrc file
$ pylint --generate-rcfile > .pylintrc

Open the .pylintrc file to edit

# open with vim editor
$ vim .pylintrc

Add lxml to extension-pkg-whitelist

# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=lxml

If you now perform the pylint test again, no error relating to lxml should appear.

JMeter and Taurus

Taurus from Blazemeter seems really to have a potential to be a star. There are new and easy ways to ease the workflow with JMeter. It allows, for example, configuration options and reports which JMeter does not offer by default.

Installation

# install via pip
$ sudo pip install bzt

On error look for installed libxml2 and libxslt libraries!

Usage

├── example.jmx
├── example.yml
└── requirements.txt
bzt==0.5.0

You don`t need jMeter or jMeter PlugIns installed! All will automatically downloaded in given path (see YAML).

---
modules:
  jmeter:
    path: ~/.bzt/jmeter-taurus/bin/jmeter
    download-link: http://apache.claz.org/jmeter/binaries/apache-jmeter-{version}.zip
    version: 2.13

execution:
  scenario:
    script: example.jmx
    variables:
      MY_TARGET_HOST: softwaretester.info

reporting:
  - module: console
  - module: final_stats
    summary: true
    percentiles: true
    test-duration: true
  - module: junit-xml
    filename: report/report.xml
    data-source: sample-labels

example.jmx

Create a JMeter testplan with “User Defined Variables”, one “Thread” with one “HTTP Request Defaults” and some “HTTP Requests”.

Taurus jMeter Example 1

On “User Defined Variables” – “Name” insert “MY_TARGET_HOST” this value will be set by Taurus YAML file.

Taurus jMeter Example 2

On “HTTP Request Defaults” – “WebServer” use the variable (MY_TARGET_HOST).

Taurus jMeter Example 3

Running JMeter test

# running headless by yaml file
$ bzt example.yml

# running headless by jmx file
$ bzt example.jmx

# running with JMeter GUI by yaml file
$ bzt example.yml -gui

# running with JMeter GUI by jmx file
$ bzt example.jmx -gui

Two folders will created on each test run. “report” (configured in YAML) and “Artifacts” (as Date/Time string). Attention – report.xml will replaced on each run!

Collecting Skype information with own python package

This time i will present 2 tutorials in one. One part describe how to create a simple Python package. The other part gives security testers a hint for sensible data. It is recommended to work with python virtualenv!

Preconditions

  • Python 2.7.x
  • pip, virtualenv, setuptools
  • Skype

Background

Skype stores sensible data, unencrypted, in a simple sqlite database (main.db). You would be surprised what information can be found there!

Example Locations

  • Mac OS – /Users/Library/Application Support/Skype/main.db
  • Windows – C:\Documents and Settings\Application Data\Skype\main.db

Python Package

.
├── MANIFEST.in
├── README.rst
├── SkypeSpy
│   └── __init__.py
└── setup.py
# -*- coding: utf-8 -*-
from setuptools import setup


def readme():
    with open('README.rst') as f:
        return f.read()

setup(
    name='SkypeSpy',
    version='1.0.0',
    description='Read values from Skype sqlite',
    long_description=readme(),
    url='<domain>',
    author='<author>',
    author_email='<email>',
    license='<license>',
    packages=['SkypeSpy'],
    include_package_data=True
)
SkypeSpy
--------

To use (with caution), simply do::

    >>> from SkypeSpy import SkypeInformation
    >>> SkypeInformation.set_db_path('path')
    >>> print SkypeInformation.get_accounts()
    >>> print SkypeInformation.get_contacts()
include README.rst
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sqlite3


class SkypeInformation(object):

    __DATABASE_PATH = str()

    @staticmethod
    def set_db_path(db_path):

        SkypeInformation.__DATABASE_PATH = str(db_path)

    @staticmethod
    def __read_from_db(sql_statement):
        """
        Read testsuite from sqlite file

        @type sql_statement: string
        @param sql_statement: sqlite select statement
        @return: list
        """

        db = sqlite3.connect(SkypeInformation.__DATABASE_PATH)
        statement = str(sql_statement)

        try:
            cursor = db.cursor()
            cursor.execute(statement)
            values = cursor.fetchall()
        except sqlite3.Error:
            values = list()
        finally:
            db.close()

        return values

    @staticmethod
    def get_accounts():

        statement = """SELECT DISTINCT
                       liveid_membername, skypename, fullname, gender,
                       languages, country, province, city, phone_home,
                       phone_office, phone_mobile, emails, homepage
                       FROM Accounts;"""

        return SkypeInformation.__read_from_db(statement)

    @staticmethod
    def get_contacts():

        statement = """SELECT DISTINCT
                       skypename, fullname, gender, languages, country,
                       province, city, phone_home, phone_office, phone_mobile,
                       emails, homepage
                       FROM Contacts;"""

        return SkypeInformation.__read_from_db(statement)

Install and execute

You can now create another environment (with virtualenv) and install the package.

# install via pip
$ pip install file:///path/to/SkypeSpy
#!/usr/bin/env python
import os
from SkypeSpy import SkypeInformation


def run():
    my_path = '/path/to/main.db'
    if os.path.exists(my_path):
        SkypeInformation.set_db_path(my_path)
        print SkypeInformation.get_contacts()
        print SkypeInformation.get_accounts()

if __name__ == '__main__':
    run()

More

There are other tables with information! Expand the package as desired.

Recommendation for JetBrains PyCharm PlugIns

This time a few recommendations for PyCharm PlugIns.

  • CodeGlance – a code mini-map similar to Sublime
  • BashSupport – Bash language support with many features
  • .ignore – PlugIn for Git, Mercurial, Docker, Chef, CVS, TeamFoundation and etc.
  • Dummy Text Generator – random text generator
  • Ini4Idea – *.ini file support
  • Markdown (Markdown support, MultiMarkdown) – *.md file support
  • IntelliBot – RobotFramework support
  • Gerrit – Gerrit Code Review integration
  • Jenkins Control Plugin – watch and trigger Jenkins builds

Create your own REST testing application

This time i want show you how to create your own REST testing application. For this demonstration i use Python, Tkinter and some Python libraries. At the end of this tutorial you can extend the application with more features like “show headers”, “store requests/responses”, “run automatically” and so on.

Preparation

  • install Python > 2.7 (Tkinter included)
  • install Requests: HTTP for Humans
  • install Python Data Validation for Humans
requests==2.8.1
validators==0.9

Example

# -*- coding: utf-8 -*-
from Tkinter import Tk, FALSE


class BaseTkGui(object):
    """Define basic TK GUI"""

    def __init__(self, window_title, window_resizable):
        """Constructor for Tkinter GUI"""

        self._root = Tk()
        self._root.title(str(window_title))

        if not bool(window_resizable):
            self._root.resizable(width=FALSE, height=FALSE)
        else:
            self._root.columnconfigure(0, weight=1)
            self._root.rowconfigure(0, weight=1)

    def start_app(self):
        """Start TK loop"""

        self._root.mainloop()

    def quit_app(self):
        """Stop and quit application"""

        self._root.quit()
# -*- coding: utf-8 -*-
from Tkinter import (Frame, OptionMenu, Entry, Button, StringVar, Label,
                     W, E, NO, END, SOLID)
from ttk import Treeview, Separator
from ScrolledText import ScrolledText
from BaseGui import BaseTkGui


class ApplicationTkGui(BaseTkGui):
    """Define application TK GUI"""

    OPTIONS = ['GET', 'POST', 'PUT', 'DELETE', 'HEAD', 'OPTIONS']

    def __init__(self, window_title):
        """Constructor for specific application GUI"""

        BaseTkGui.__init__(self, window_title, False)

        self._method = None
        self._url = None
        self._tree = None
        self._status = None
        self._time = None
        self._key = None
        self._value = None
        self._output = None

    def build_frames(self):
        """Add all frames and start mainloop"""

        self.__top_frame()
        self.__middle_frame()
        self.__bottom_frame()

        self.start_app()

    def prepare_req(self):
        """Overwritten method"""

        pass

    def _add_items(self):
        """Add items into TreeView at end"""

        key = str(self._key.get())
        value = str(self._value.get())

        if key and value:
            self._tree.insert("", "end", values=(key, value))
            self._key.delete(0, END)
            self._value.delete(0, END)

        self._tree.bind("<Double-1>", self._delete_item)

    def _delete_item(self, event):
        """Delete items from TreeView by ID"""

        item = self._tree.identify_row(event.y)
        self._tree.delete(item)

    def __top_frame(self):
        """Top frame creator"""

        self._method = StringVar(self._root)
        self._method.set("GET")

        frame = Frame(self._root)
        frame.grid(column=0, row=0, sticky=W+E)
        frame.grid_rowconfigure(0, weight=1)
        frame.grid_columnconfigure(0, weight=1)

        option = OptionMenu(frame, self._method, *self.OPTIONS)
        option.grid(column=0, row=0, padx=5, pady=5)

        self._url = Entry(frame, width=50)
        self._url.grid(column=1, row=0, padx=5, pady=5)
        self._url.configure(borderwidth=1, relief=SOLID)
        self._url.configure(highlightthickness=0)
        self._url.insert(0, 'http://')

        submit = Button(frame, text='Submit', command=self.prepare_req)
        submit.grid(column=3, row=0, padx=5, pady=5)

        Separator(frame).grid(columnspan=4, row=1, sticky=W+E)

    def __middle_frame(self):
        """Middle frame creator"""

        frame = Frame(self._root)
        frame.grid(column=0, row=1, sticky=W+E)
        frame.grid_rowconfigure(0, weight=1)
        frame.grid_columnconfigure(0, weight=1)

        self._tree = Treeview(frame, columns=("Key", "Val"), selectmode='none')
        self._tree.grid(columnspan=5, row=0, padx=5, pady=5, sticky=W+E)
        self._tree.column('#0', stretch=NO, minwidth=0, width=0)
        self._tree.heading('#1', text='Key')
        self._tree.heading('#2', text='Value')
        self._tree.configure(height=5)

        key = Label(frame, text='Key:')
        key.grid(column=0, row=1, padx=5, pady=5, sticky=E)
        self._key = Entry(frame)
        self._key.grid(column=1, row=1, padx=5, pady=5, sticky=W)
        self._key.configure(borderwidth=1, relief=SOLID)
        self._key.configure(highlightthickness=0)

        value = Label(frame, text='Value:')
        value.grid(column=2, row=1, padx=5, pady=5, sticky=E)
        self._value = Entry(frame)
        self._value.grid(column=3, row=1, padx=5, pady=5, sticky=W)
        self._value.configure(borderwidth=1, relief=SOLID)
        self._value.configure(highlightthickness=0)

        add = Button(frame, text='Add new header', command=self._add_items)
        add.grid(column=4, row=1, padx=5, pady=5)

        Separator(frame).grid(columnspan=5, row=2, sticky=W+E)

    def __bottom_frame(self):
        """Bottom frame creator"""
        font_style = ('verdana', 10, 'normal')

        frame = Frame(self._root)
        frame.grid(column=0, row=2, sticky=W+E)
        frame.grid_rowconfigure(0, weight=1)
        frame.grid_columnconfigure(0, weight=1)

        self._status = Label(frame, text='Response Status: -')
        self._status.grid(column=0, row=0, sticky=W)
        self._status.configure(fg='red', font=font_style)

        self._time = Label(frame, text='Time: - ms')
        self._time.grid(column=1, row=0, sticky=E)
        self._time.configure(fg='red', font=font_style)

        self._output = ScrolledText(frame)
        self._output.grid(columnspan=2, row=1, padx=5, pady=5, sticky=W+E)
        self._output.configure(height=12, borderwidth=1, relief=SOLID)
        self._output.configure(highlightthickness=0)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
import validators
from Tkinter import END
from ApplicationGui import ApplicationTkGui


class Application(ApplicationTkGui):
    """Define application behaviors"""

    def __init__(self, window_title):
        """Constructor for application with title"""

        ApplicationTkGui.__init__(self, window_title)

    def run_application(self):
        """Start application"""

        self.build_frames()

    def prepare_req(self):
        """Prepare and validate request"""

        headers = dict()
        method = self._method.get()
        url = self._url.get()
        tree_ids = self._tree.get_children()

        for i in tree_ids:
            row = self._tree.item(i)
            add = row['values']
            headers[add[0]] = add[1]

        if validators.url(url):
            self.__do_request(method, url, headers)

    def __do_request(self, method, url, headers):
        """Do request and print results"""

        self._output.delete(1.0, END)

        ses = requests.Session()
        prepped = requests.Request(method, url, headers).prepare()
        response = ses.send(prepped, verify=False, allow_redirects=True)

        code = response.status_code
        body = response.text
        time_delta = response.elapsed
        duration = time_delta.total_seconds()

        status = 'Response Status: ' + str(code)
        time = 'Time: ' + str(duration) + ' ms'
        text = body[0:900] + ' ...'

        self._status.configure(text=status)
        self._time.configure(text=time)
        self._output.insert(END, text)

if __name__ == '__main__':

    RUN = Application('HTTPTester')
    RUN.run_application()

Run application

# set access permissions
$ chmod u+x ./RESTme.py
 
# start application
$ python -B ./RESTme.py

The application should look like:

RESTme Application

Create your own fake-data generator application

In this episode i want to show you, how easy it is to create a fake-data generator application.

Why?

  1. to get inspired
  2. if you need such tool behind restricted areas (without internet)
  3. software testers need a lot of fake-data

Required

  • Python > 2.7
  • Python Faker package installed
# install by pip
$ sudo pip install fake-factory

Example Application Code

# -*- coding: utf-8 -*-
from faker import Factory


class BasicGenerator(object):

    SETTINGS = ('Prefix', 'Name', 'Address', 'E-Mail', 'Phone',
                'Birthday', 'Company')

    def __init__(self):
        """Constructor for vars and factory object"""
        self._repeat_count = None
        self._selection = None
        self.__faker = Factory.create()

    def faker_data(self, count, selection):
        """Return fake data string"""
        output = str()
        loops = int(count)
        value = tuple(selection)
        for i in range(0, loops):
            if 0 in value:
                output += self.__faker.prefix()
            if 1 in value:
                output += self.__faker.name() + '\n'
            if 2 in value:
                output += self.__faker.address() + '\n'
            if 3 in value:
                output += self.__faker.email() + '\n'
            if 4 in value:
                output += self.__faker.phone_number() + '\n'
            if 5 in value:
                output += self.__faker.date() + '\n'
            if 6 in value:
                output += self.__faker.company() + '\n'
            output += '\n'
        return output
# -*- coding: utf-8 -*-
from BasicGenerator import BasicGenerator
from Tkinter import (Tk, Button, Variable, Listbox, MULTIPLE, FALSE, END,
                     Spinbox, E, W)
from ScrolledText import ScrolledText


class FakeNameGenerator(BasicGenerator):

    def __init__(self):
        """Constructor for Tkinter GUI"""
        BasicGenerator.__init__(self)
        self.__root = Tk()
        self.__root.title('FakeNameGenerator')
        self.__root.resizable(width=FALSE, height=FALSE)
        self.__result_output = None
        self.__box_opt = Variable(self.__root)
        self.__box_opt.set(self.SETTINGS)

    def start_app(self):
        """Build widgets and start TK loop"""
        self.__control_widget()
        self.__output_widget()
        self.__root.mainloop()

    def quit_app(self):
        """Stop and close application"""
        self.__root.quit()

    def __reset_output(self):
        """Reset TK text"""
        self.__result_output.delete(1.0, END)

    def __run_fake_data(self):
        """Validate and generate output"""
        count = int(self._repeat_count.get())
        selection = self._selection.curselection()
        if not selection:
            self.__result_output.insert(END, 'No value selected!\n\n')
        elif count < 1:
            self.__result_output.insert(END, 'Min. 1 need!\n\n')
        elif count > 10:
            self.__result_output.insert(END, 'Max. 10 allowed!\n\n')
        else:
            data = self.faker_data(count, selection)
            self.__result_output.insert(END, data)

    def __control_widget(self):
        """Build control widgets"""
        self._selection = Listbox(self.__root,
                                  listvariable=self.__box_opt,
                                  selectmode=MULTIPLE)
        self._selection.grid(padx=5, column=1, row=1, sticky=W+E)
        self._repeat_count = Spinbox(self.__root,
                                     from_=1,
                                     to=10)
        self._repeat_count.grid(column=1, row=2, sticky=W+E)
        Button(self.__root,
               text='Create',
               command=self.__run_fake_data).grid(column=1, row=3, sticky=W+E)
        Button(self.__root,
               text='Reset',
               command=self.__reset_output).grid(column=1, row=4, sticky=W+E)
        Button(self.__root,
               text='Quit',
               command=self.quit_app).grid(column=1, row=6, sticky=W+E)

    def __output_widget(self):
        """Build output widget"""
        self.__result_output = ScrolledText(self.__root, width=80)
        self.__result_output.grid(padx=5, pady=5, column=0, row=0, rowspan=7)
        self.__result_output.config(borderwidth=1, highlightthickness=1)


if __name__ == '__main__':
    RUN = FakeNameGenerator()
    RUN.start_app()

Run Fake-Data Application

# set access permissions
$ chmod u+x FakeNameGenerator.py

# start application
$ python -B ./FakeNameGenerator.py

Now it`s on you to improve this tiny application! As example you could add file and/or database export, add more functionality or just improve the GUI.

Jenkins and Virtualenv

This guide is intended to show how you can use Jenkins/Hudson with Python virtualenv.

Precondition

Preparation

# install on Debian/Ubuntu/Mint
$ sudo aptitude update
$ sudo aptitude install python-pip python-virtualenv

# install on Mac OS
$ sudo easy_install pip
$ sudo pip install virtualenv

Example

Create (if necessary) a new “Freestyle Project” and configure as needed build-paramaters , VCS and etc. On section “Build” – “Execute Shell” insert following script.

# set variable
ExampleENV="${WORKSPACE}"/.ExampleENV

# delete folder and content if exists
if [ -d "$ExampleENV" ]; then
	rm -fr "$ExampleENV"
fi

# create new virtualenv
virtualenv --no-site-packages "$ExampleENV"

# activate virtualenv
. "$ExampleENV"/bin/activate

# CODE FOR VIRTUALENV...

Create PDF test-reports with Python

With only two python libraries and some line of codes, you are able to create very nice test-reports in PDF format. This tutorial should give you a hint. Feel free to improve with your requirements.

Preconditions

Example

matplotlib==1.4.3
reportlab==3.2.0
# -*- coding: utf-8 -*-

import time
import matplotlib.pyplot as plt

from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, Image
from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
from reportlab.lib.enums import TA_JUSTIFY
from reportlab.lib.pagesizes import letter
from reportlab.lib.units import inch


class CreateTestReport(object):
    """Example test report class"""

    def __init__(self, test_results, file_name):
        """Constructor method"""
        self.data = test_results
        self.file_name = file_name

        self.story = list()

        self.styles = getSampleStyleSheet()
        self.styles.add(ParagraphStyle(name='Justify', alignment=TA_JUSTIFY))

    def __create_overview(self):
        """Create overview graphic and pdf content"""
        requests = list()
        duration = list()

        for lists in self.data:
            requests.append(lists[0])
            duration.append(lists[1])

        plt.plot(duration)

        plt.xlabel('http requests')
        plt.ylabel('response time')
        plt.xticks(range(len(duration)), requests)

        plt.grid(True)
        plt.savefig('overview.png')

        text = '<font size=18>Overview</font>'
        self.story.append(Paragraph(text, self.styles["Heading2"]))
        picture = Image('overview.png', 6*inch, 4*inch, hAlign='LEFT')
        self.story.append(picture)

        text = '<font size=12>Lorem ipsum dolor sit amet, <br/>\
        consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt \
        ut labore et dolore magna aliquyam erat, sed diam voluptua.<br/>\
        At vero eos et accusam et justo duo dolores et ea rebum. <br/>\
        Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum \
        dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing \
        elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore \
        magna aliquyam erat, sed diam voluptua. <br/>\
        At vero eos et accusam et justo duo dolores et ea rebum. <br/>\
        Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum \
        dolor sit amet.</font>'
        self.story.append(Paragraph(text, self.styles["Normal"]))

        self.story.append(Spacer(1, 192))

    def __create_round_trips(self):
        """Create round trip graphics and pdf content"""

        for lists in self.data:
            text = '<font size=18>%s: %s HTTP/1.1</font>' % (lists[3],
                                                             lists[4])
            self.story.append(Paragraph(text, self.styles["Heading2"]))

            text = '<font size=12>Status Code: %s</font>' % lists[2]
            self.story.append(Paragraph(text, self.styles["Normal"]))
            self.story.append(Spacer(1, 12))

            file_name = 'request_' + str(lists[0]) + '.png'
            title = 'Request Circle (' + str(lists[1]) + ' sec.)'
            plt.figure(figsize=(4, 4))
            plt.title(title)
            values = lists[5]
            labels = ['Request', 'Server', 'Response']
            plt.pie(values, labels=labels)
            plt.savefig(file_name)

            picture = Image(file_name, 2.5*inch, 2.5*inch, hAlign='LEFT')
            self.story.append(picture)

            self.story.append(Spacer(1, 24))

    def create_pdf(self, report_title, author, base_url, margins):
        """Create PDF document"""

        formatted_time = time.ctime()
        pdf = SimpleDocTemplate(self.file_name,
                                pagesize=letter,
                                title=report_title,
                                author=author,
                                topMargin=margins[0],
                                leftMargin=margins[1],
                                rightMargin=margins[2],
                                bottomMargin=margins[3])

        # Add headline, created date/time and author
        text = '<font size=24>%s</font>' % report_title
        self.story.append(Paragraph(text, self.styles["Heading1"]))
        self.story.append(Spacer(1, 12))

        text = '<font size=10>%s<br/>by %s<br/>%s</font>' % (formatted_time,
                                                             author,
                                                             base_url)
        self.story.append(Paragraph(text, self.styles["Italic"]))
        self.story.append(Spacer(1, 24))

        # Add overview
        self.__create_overview()

        # Add results
        self.__create_round_trips()

        # Create and store report
        pdf.build(self.story)


if __name__ == '__main__':
    # some test result data
    data = ([1, 0.5, 200, 'GET', '/api/getUser', [40, 40, 20]],
            [2, 0.75, 200, 'POST', '/api/createUser', [50, 30, 20]],
            [3, 0.25, 404, 'PUT', '/api/updateUser', [50, 25, 25]],
            [4, 2, 301, 'GET', '/api/Logout', [25, 50, 25]],
            [5, 0.25, 200, 'POST', '/api/Login', [45, 35, 20]])

    # create report
    CREATE = CreateTestReport(data, 'TestReport.pdf')
    CREATE.create_pdf('Example Test Report',
                      'Lupin3000',
                      'http://example.com',
                      [20, 20, 20, 10])

You can call it simply by

# execute without compile
$ python -B ExampleReport.py