Simple Doctests with PyCharm CE

Python Doctests with PyCharm are very easy to configure! This tutorial will show you – how easy you can configure and run your Doctests inside PyCharm CE. You can use the following pyton script.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
This is an example for python doctest inside module docstring

>>> add('i', 'i')
Traceback (most recent call last):
    ...
TypeError: can't multiply sequence by non-int of type 'str'

"""


def add(a, b):
    """
    This is an example for python doctest inside function docstring

    >>> add(2, 3)
    6
    >>> add('a', 3)
    'aaa'
    """
    return a * b


class SomeTest(object):
    """
    This is an example for python doctest inside class docstring

    >>> t = SomeTest(); t.add(2, 'b'); t.sum
    'bb'
    """

    def __init__(self):
        """
        This is an example for python doctest inside constructor docstring

        >>> t = SomeTest(); type(t.sum)
        <type 'int'>
        """
        self.sum = int()

    def add(self, a, b):
        """
        This is an example for python doctest inside method docstring

        >>> t = SomeTest(); t.add(5, 5); t.sum
        25
        >>> t = SomeTest(); t.add('a', 5); t.sum
        'aaaaa'
        """
        self.sum = a * b

Now create following Doctests for Script, Class, Method and Function.

Script

pycharm doctest for script

Class

pycharm doctest for class

Method

pycharm doctest for method

Function

pycharm doctest for function

Now you can run your different doctests and look on results.

pycharm doctest results example

PyCharm – TERM environment variable not set

It can happen that you get this message in the PyCharm console. “TERM environment variable not set.” Here now the simple way to solve that issue.

The example Python script

#!/usr/bin/env python
# -*- coding: utf8 -*-

import os

os.system('clear')

The annoying error will displayed in PyCharm.

Solution

Open “Run/Debug configuration” and add an environment variable “TERM=xterm-color”

PyCharm environment variable
PyCharm run debug configuration

That’s it already … The message should no longer appear.

Python, Selenium Grid and Docker

With Docker you can quickly and easily install, configure and use Selenium Grid. This tutorial shows the respective steps that you need as a software tester (or Developer). Instead of Python you can also use other languages, which are supported by Selenium​.

Preconditions

Preparation of files

# create new project
$ mkdir -p ~/Project/SeleniumTutorial && cd ~/Project/SeleniumTutorial

# create docker-compose.yml (version 1)
$ vim v1-docker-compose.yml

# or create docker-compose.yml (version 2)
$ vim v2-docker-compose.yml

# create python example.py
$ vim example.py

Note: You can opt for a version of docker-compose.yml!

Version: 1

---
selenium_hub:
  image: selenium/hub
  ports:
    - 4444:4444
node_1:
  image: selenium/node-chrome
  links:
    - selenium_hub:hub
node_2:
  image: selenium/node-firefox
  links:
    - selenium_hub:hub

Version: 2

---
version: '2'
services:
  selenium_hub:
    image: selenium/hub
    ports:
      - 4444:4444
  node_1:
    image: selenium/node-chrome
    depends_on:
      - selenium_hub
    environment:
      - HUB_PORT_4444_TCP_ADDR=selenium_hub
  node_2:
    image: selenium/node-firefox
    environment:
      - HUB_PORT_4444_TCP_ADDR=selenium_hub
    depends_on:
      - selenium_hub
import os
import datetime
import time
import unittest
from selenium import webdriver


class Example(unittest.TestCase):

    def setUp(self):

        self.driver = webdriver.Remote(
            command_executor='http://192.168.99.100:4444/wd/hub',
            desired_capabilities={
                'browserName': 'firefox',
                'javascriptEnabled': True
            }
        )

        self.driver.get('http://softwaretester.info/')

    def test_something(self):

        dt_format = '%Y%m%d_%H%M%S'
        cdt = datetime.datetime.fromtimestamp(time.time()).strftime(dt_format)
        current_location = os.getcwd()
        img_folder = current_location + '/images/'

        if not os.path.exists(img_folder):
            os.mkdir(img_folder)

        picture = img_folder + cdt + '.png'
        self.driver.save_screenshot(picture)

    def tearDown(self):

        self.driver.quit()


if __name__ == "__main__":

    unittest.main(verbosity=1)

Create environment

# create new VM
$ docker-machine create -d virtualbox Grid

# pointing shell
$ eval $(docker-machine env Grid)

# show status (optional)
$ docker-machine ls
...
NAME   ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
Grid   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.11.1 

# run docker-compose (Version: 1)
$ docker-compose -f v1-docker-compose.yml up -d

# run docker-compose (Version: 2)
$ docker-compose -f v2-docker-compose.yml up -d

# show status (Version: 1)
$ docker-compose -f v1-docker-compose.yml ps
...
             Name                         Command           State           Ports          
------------------------------------------------------------------------------------------
seleniumtutorial_node_1_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_node_2_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_selenium_hub_1   /opt/bin/entry_point.sh   Up      0.0.0.0:4444->4444/tcp

# show status (Version: 2)
$ docker-compose -f v2-docker-compose.yml ps
...
             Name                         Command           State           Ports          
------------------------------------------------------------------------------------------
seleniumtutorial_node_1_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_node_2_1         /opt/bin/entry_point.sh   Up                           
seleniumtutorial_selenium_hub_1   /opt/bin/entry_point.sh   Up      0.0.0.0:4444->4444/tcp

Open Browser

Selenium Grid Console

Run Python script

# run python selenium script
$ python -B ~/Projects/Selenium/example.py

Note: Via browserName (example.py) you can choose the respective browser (firefox or chrome)!

Note: Via docker-compose scale you can add/remove node instances!

# create 2 instances (Version: 1)
$ docker-compose -f v1-docker-compose.yml scale node_1=2

# create 3 instances (Version: 2)
$ docker-compose -f v2-docker-compose.yml scale node_2=3

Create REST API mock server with Docker

This time again a tutorial with various instructions. It is a REST API services for development and testing purposes and some simple Docker instructions.

Preconditions

Note: For Mac OS X and Windows use Docker Toolbox!

Create and connect into Boot2Docker VM

# create new boot2docker vm
$ docker-machine create -d virtualbox apivm

# list created vm(s)
$ docker-machine ls

# list informations (optional)
$ docker-machine inspect apimock

# ssh into boot2docker vm
$ docker-machine ssh apivm

Create Dockerfile (inside VM)

# create new Dockerfile
$ vi Dockerfile
FROM ubuntu

# install python packages
RUN apt-get update && apt-get -y install python python-dev python-pip

# install python libraries
RUN pip install mock-server tornado==4.2

# create directory
RUN mkdir -p api

EXPOSE 8888

CMD ["mock-server","--address=0.0.0.0","--dir=api"]

Create Docker image and container (inside VM)

# create Docker image
$ docker build -t api_image .

# list Docker image(s)
$ docker images

# create and start new container from Docker image
$ docker run --name api_container -d -p 8888:8888 api_image

# list Docker container(s)
$ docker ps -a

Run application in browser

Now open a browser and call URL like: http://<192.168.99.100>:8888/__manage. You can now begin to create and use REST API resources.

Multiple hosts provisioning with Vagrant, Ansible and virtualenv

In this tutorial we use Ansible (installed in virtualenv) and Vagrant. Furthermore, we have different machines (Debian, CentOS). For all hosts we want to have Provisioning on startup and via command.

Precondition

Folder structure

.
├── Makefile
├── Vagrantfile
├── playbook.yml
├── requirements.txt
└── roles
    └── common
        └── tasks
            └── main.yml

Files

ansible
ansible-lint
VAGRANTFILE_API_VERSION = "2"
 
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # CentOS 7 VM
  config.vm.define "centos" do |centos|
    centos.vm.box = "lupin/centos7"
    centos.vm.network :forwarded_port, guest: 22, host: 2221, id: 'ssh'
    centos.vm.provider "virtualbox" do |vb|
       vb.name = "CentOS7-Vagrant-Ansible"
    end
    centos.vm.provision "ansible" do |ansible|
        # ansible.verbose = "v"
        ansible.playbook = "playbook.yml"
    end
  end

  # Debian 8 VM
  config.vm.define "debian" do |debian|
    debian.vm.box = "lupin/debian8"
    debian.vm.network :forwarded_port, guest: 22, host: 2222, id: 'ssh'
    debian.vm.provider "virtualbox" do |vb|
       vb.name = "Debian-Vagrant-Ansible"
    end
    debian.vm.provision "ansible" do |ansible|
        # ansible.verbose = "v"
        ansible.playbook = "playbook.yml"
    end
  end

end
---
- hosts: all
  become: yes
  gather_facts: yes
  roles:
    - common
---
- debug: msg="System {{ ansible_distribution }}"
ENV_DIR = env
CURRENT_DIR := $(shell pwd)
INTERPRETER = $(CURRENT_DIR)/$(ENV_DIR)/bin/
PATH := ${PATH}:$(INTERPRETER)

help:
	@echo "Run make <target> with:"
	@echo " > env           : create virtualenv on folder $(ENV_DIR)"
	@echo " > deps          : install dependentcies"
	@echo " > cleanenv      : delete virtualenv"
	@echo " > start         : run vagrant up"
	@echo " > provisioning  : start ansible provisioning"
	@echo " > kill          : run vagrant destroy"

debug:
	@echo " > ansible location is     : $(INTERPRETER)"
	@echo " > environment variable is : $(PATH)"
	vagrant status

env:
	virtualenv $(ENV_DIR) && \
	. $(ENV_DIR)/bin/activate && \
	make deps

deps:
	$(ENV_DIR)/bin/pip install -r requirements.txt

cleanenv:
	rm -fr $(ENV_DIR)

start:
	vagrant up

provisioning:
	vagrant provision

kill:
	vagrant destroy -f

Usage

# create environment
$ make env

# start vagrant (create VM`s and run provisioning)
$ make start

# run provisioning (on started VM`s)
$ make provisioning

# stop vagrant (delete VM`s)
$ make kill

# delete environment
$ make cleanenv

Hint

Check out the by Vagrant generated inventory file!

$ cat .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory

Python profiling with PyCharm Community Edition

Before we start, if you don`t know what is profiling read this Wikipedia article! In my opinion profiling should be a part of every development/build process! Whether the responsibility lies with QA or development. Python profiler are supported only in PyCharm Professional Edition. This article show you the possibilities for the community edition.

Preparation

  • PyCharm installed
  • Virtualenv or similar installed (optional)
  • PyCharm BashSupport Plugin installed

The easiest Profiler

With Unix/Linux time command you have allready a simple profiler! Time writes a message to standard output. Here you will find some information on Stackoverflow.

#!/usr/bin/env python
# -*- coding: utf-8 -*-


def hello_world():

    for i in range(1, 5):
        print '%d Hello world from python...' % i


if __name__ == '__main__':
    hello_world()

With BashSupport Plugin we can setup the “Run/Debug Configuration” like:

unix time profiler

Better informations

But now we need better information. For this we use cProfile, cprofilev and snakeviz.

# cProfile is part of standard python library

# install snakeviz
$ pip install snakeviz

# install cprofildev
$ pip install cprofilev

“Run/Debug Configuration” example

cProfile simple

Now will store the results into a file

cProfile store output

With snakeviz you can open the profile in browser:

$ snakeviz output.prof

The other option is to use cprofilev:

cprofilev

Even more information

If that was not enough,… we install some more libraries.

# install line_profiler
$ pip install line_profiler

# install memory_profiler and psutil
$ pip install memory_profiler
$ pip install psutil

Now we need to change the example code. We add the decorator…

#!/usr/bin/env python
# -*- coding: utf-8 -*-


@profile
def hello_world():

    for i in range(1, 5):
        print '%d Hello world from python...' % i


if __name__ == '__main__':
    hello_world()

the line_profiler configuration

kernprofiler

the memory_profiler

memory profiler

All configurations could now startet via the “Run” button. There are even more Profiler that you can use with similar PyCharm.

Ansible and PyCharm

Of course you can run Ansible within PyCharm via command-line, but it also works with the “Run” button.

Preparation

  • PyCharm project created (maybe with virtualenv)
  • YAML/Ansible support Plugin installed (optional)
  • BashSupport Plugin installed

Configuration

Open “Run/Debug Configurations” and add new Bash configuration. Give a name and Script value. The value should be the main Ansible playbook. As Interpreter path value select the ansible-playbook binary. For Interpreter option insert the Ansible inventory file. The last value is your current working directory. If you don’t set this value, the values for playbook and inventory need configured with absolute path!

PyCharm and Ansible

Now you can create different configurations and run Ansible via “Run” button.

Simple port scanner with Python

If you like Python and NMap … there is a very good wrapper from Alexandre Norman! This tutorial show a very simple example for usage.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import nmap


def port_scan(target, ports):
    """
    Simple NMap port scanner example

    @param target: host for scan
    @type target: string
    @param ports: ports for scan
    @type ports: string
    """

    nmap_scan = nmap.PortScanner()
    nmap_scan.scan(str(target), str(ports))

    for host in nmap_scan.all_hosts():
        print '=' * 80
        print 'Host:\t%s' % host
        print 'State:\t%s\n' % nmap_scan[host].state()

        for protocol in nmap_scan[host].all_protocols():
            print 'Protocol(s): %s' % protocol

            port_list = list(nmap_scan[host][protocol].keys())
            port_list.sort()

            for port in port_list:
                print '\n[+] Port: %s' % port
                print '[+] State: %s' % nmap_scan[host][protocol][port]


if __name__ == '__main__':
    port_scan('192.168.192.1', '1-1000')

For running see following lines:

# change execution
$ chmod u+x example.py

# start script
$ python -B ./example.py