Monitor running docker containers with cAdvisor

As a software tester, you have several containers run in your environment. Here is an example how easily and quickly you can monitor your test-environment with cAdvisor.

Preconditions

Preparation

# create project (local)
$ mkdir -p ~/Projects/Monitoring && cd ~/Projects/Monitoring

# create shell script (local)
$ vim start-demo.sh
#!/usr/bin/env sh

docker run -d --name cadvisor -P -v /:/rootfs:ro -v /var/run:/var/run:rw -v /sys:/sys:ro -v /var/lib/docker/:/var/lib/docker:ro google/cadvisor:latest
docker run -d --name jenkins -P jenkins
docker run -d --name selenium-hub -P selenium/hub:2.53.0
docker run -d --name selenium-node_1 --link selenium-hub:hub selenium/node-chrome:2.53.0
docker run -d --name selenium-node_2 --link selenium-hub:hub selenium/node-firefox:2.53.0

Note: You can also assign the respective ports!

Run docker containers

# create new VM (local)
$ docker-machine create -d virtualbox monitor

# show status (local)
$ docker-machine ls
...
NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
monitor   -        virtualbox   Running   tcp://192.168.99.100:2376           v1.11.1  

# copy into VM (local) 
$ docker-machine scp ~/Projects/Monitoring/start-demo.sh monitor:/home/docker/

# ssh into VM (local into VM)
$ docker-machine ssh monitor

# change rights (VM)
$ chmod +x start-demo.sh && ls -la

# run shell script (VM)
$ ./start-demo.sh

# list running docker container (VM)
$ docker ps -a
...
CONTAINER ID        IMAGE                          COMMAND                  CREATED              STATUS              PORTS                                               NAMES
57c2598b4261        selenium/node-firefox:2.53.0   "/opt/bin/entry_point"   4 seconds ago        Up 4 seconds                                                            selenium-node_2
d79a5123bcfc        selenium/node-chrome:2.53.0    "/opt/bin/entry_point"   29 seconds ago       Up 29 seconds                                                           selenium-node_1
095f9844346d        selenium/hub:2.53.0            "/opt/bin/entry_point"   About a minute ago   Up About a minute   0.0.0.0:32771->4444/tcp                             selenium-hub
8db3ad58d8ce        jenkins                        "/bin/tini -- /usr/lo"   About a minute ago   Up About a minute   0.0.0.0:32770->8080/tcp, 0.0.0.0:32769->50000/tcp   jenkins
d1e1e1c36d6d        google/cadvisor:latest         "/usr/bin/cadvisor -l"   2 minutes ago        Up 2 minutes        0.0.0.0:32768->8080/tcp                             cadvisor

Open browser

cAdvisor

docker-compose and Jenkins

In this tutorial i show an example, how to install Jenkins (version 2.0) via docker-compose (and docker-machine).

Preconditions

Preparation

# create example directories
$ mkdir -p ~/Projects/Example/Jenkins_HOME && cd ~/Projects/Example

# create new and edit compose files
$ vim docker-compose.yml
---
version: '2'
services:
  jenkins:
    image: jenkins:2.0
    container_name: jenkins
    restart: always
    ports:
      - 8080:8080
    volumes:
      - ./FOR_JENKINS:/var/jenkins_home

Build and run

# create new VM
$ docker-machine create -d virtualbox --virtualbox-memory "2048" example-vm

# point shell
$ eval $(docker-machine env example-vm)

# show current state
$ docker-machine ls
...
NAME         ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
example-vm   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.11.0 

# run docker-compose
$ docker-compose up -d

# show state
$ docker-compose ps
...
Name                Command               State                 Ports               
------------------------------------------------------------------------------------
jenkins   /bin/tini -- /usr/local/bi ...   Up      50000/tcp, 0.0.0.0:8080->8080/tcp

# get administrator password
$ docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Run Browser

docker jenkins container

JMeter and Taurus

Taurus from Blazemeter seems really to have a potential to be a star. There are new and easy ways to ease the workflow with JMeter. It allows, for example, configuration options and reports which JMeter does not offer by default.

Installation

# install via pip
$ sudo pip install bzt

On error look for installed libxml2 and libxslt libraries!

Usage

├── example.jmx
├── example.yml
└── requirements.txt
bzt==0.5.0

You don`t need jMeter or jMeter PlugIns installed! All will automatically downloaded in given path (see YAML).

---
modules:
  jmeter:
    path: ~/.bzt/jmeter-taurus/bin/jmeter
    download-link: http://apache.claz.org/jmeter/binaries/apache-jmeter-{version}.zip
    version: 2.13

execution:
  scenario:
    script: example.jmx
    variables:
      MY_TARGET_HOST: softwaretester.info

reporting:
  - module: console
  - module: final_stats
    summary: true
    percentiles: true
    test-duration: true
  - module: junit-xml
    filename: report/report.xml
    data-source: sample-labels

example.jmx

Create a JMeter testplan with “User Defined Variables”, one “Thread” with one “HTTP Request Defaults” and some “HTTP Requests”.

Taurus jMeter Example 1

On “User Defined Variables” – “Name” insert “MY_TARGET_HOST” this value will be set by Taurus YAML file.

Taurus jMeter Example 2

On “HTTP Request Defaults” – “WebServer” use the variable (MY_TARGET_HOST).

Taurus jMeter Example 3

Running JMeter test

# running headless by yaml file
$ bzt example.yml

# running headless by jmx file
$ bzt example.jmx

# running with JMeter GUI by yaml file
$ bzt example.yml -gui

# running with JMeter GUI by jmx file
$ bzt example.jmx -gui

Two folders will created on each test run. “report” (configured in YAML) and “Artifacts” (as Date/Time string). Attention – report.xml will replaced on each run!

Jenkins and Virtualenv

This guide is intended to show how you can use Jenkins/Hudson with Python virtualenv.

Precondition

Preparation

# install on Debian/Ubuntu/Mint
$ sudo aptitude update
$ sudo aptitude install python-pip python-virtualenv

# install on Mac OS
$ sudo easy_install pip
$ sudo pip install virtualenv

Example

Create (if necessary) a new “Freestyle Project” and configure as needed build-paramaters , VCS and etc. On section “Build” – “Execute Shell” insert following script.

# set variable
ExampleENV="${WORKSPACE}"/.ExampleENV

# delete folder and content if exists
if [ -d "$ExampleENV" ]; then
	rm -fr "$ExampleENV"
fi

# create new virtualenv
virtualenv --no-site-packages "$ExampleENV"

# activate virtualenv
. "$ExampleENV"/bin/activate

# CODE FOR VIRTUALENV...

JUnit report with Python Webdriver

To create JUnit reports, you just need the python library xmlrunner. This is needed for the integration with build server like Jenkins.

Installation

# example installation with pip
$ sudo pip install xmlrunner

Usage

Just replace for example:

unittest.TextTestRunner(verbosity=2).run(test_suite)

with

from xmlrunner import xmlrunner

""" code for test suite ... """

xmlrunner.XMLTestRunner(verbosity=2, output='reports').run(test_suite)

Jenkins

On Jenkins add a new “post-build” action and select the Publish JUnit test result report. Add now the string “reports/*.xml” into the “Test report XMLs” field.

Python, Selenium and PhantomJS – ignore certificate errors

Background

You are done with your work and push all into Git. The Build-Server starts his work and all test scripts are failing. Short look and it is clear – certificate errors. The next example shows, how to ignore certificate errors on PhantomJS.

@classmethod
def setUpClass(cls):
    """starting point for test cases"""

    cls.driver = webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true'])
    cls.driver.implicitly_wait(30)
    cls.driver.set_window_size(1024, 768)
    cls.driver.get(cls.url)

Now it should work….

Jenkins – Modern Status Plugin

Who the original symbols of Jenkins do not like, should try the Modern Status Plugin. The tiny plugin of Oliver Vinn provide a new and very cool set of icons for the continuous integration server. The installation is very simple!

Example

This is a example of original icons for Jenkins/Hudson:

jenkins original status icons

Yes they might lead to confusion and don’t look awesome.

Steps

Open the Plugin Manager and search for “Modern Status Plugin”.

jenkins modern status plugin

After restart Jenkins/Hudson the new iconset should be available and look like:

jenkins modern status

This looks quite nice and speaks much better, already on first view, about the status.

Sass, Compass and Jenkins

This tutorial shows how Sass, Compass and Jenkins working together in very easy way. The goal is that Jenkins created the CSS from SCSS files.

Precondition

  • Jenkins installed
  • Ruby installed

Steps

In first step we need to install Sass and Compass on the clients (Developer engines) and build server (Jenkins).

# check for installed libraries
$ gem list

# update gem`s
$ sudo gem update --system

# install sass and compass libraries
$ sudo gem install sass
$ sudo gem install compass

Now we create on one client the project.

# change to specific folder
$ cd ~/path/to/folder

# create compass project
$ compass create --bare --sass-dir "sass" --css-dir "css" --javascripts-dir "js" --images-dir "img"

# create folders
$ mkdir img js css sass/partials

# create files
$ touch index.html js/main.js sass/style.scss sass/partials/_reset.scss sass/partials/_content.scss

Now add and edit some content to the files with your favorite editor and commit all to your repository. The content for the tutorial looks like this:

require 'compass/import-once/activate'
# Require any additional compass plugins here.

# Set this to the root of your project when deployed:
http_path = "/"
css_dir = "css"
sass_dir = "sass"
images_dir = "img"
javascripts_dir = "js"

# You can select your preferred output style here (can be overridden via the command line):
output_style = :compressed

# To enable relative paths to assets via compass helper functions. Uncomment:
# relative_assets = true

# To disable debugging comments that display the original location of your selectors. Uncomment:
line_comments = false


# If you prefer the indented syntax, you might want to regenerate this
# project again passing --syntax sass, or you can uncomment this:
# preferred_syntax = :sass
# and then run:
# sass-convert -R --from scss --to sass sass scss && rm -rf sass && mv scss sass
<!doctype html>
<head>
	<title>Lorem ipsum</title>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width">
	<link rel="stylesheet" href="css/styles.css">
</head>
<body>
	<header>
		Header Content
	</header>
	<div id="wrapper">
		<h1>Lorem ipsum</h1>
		<article>
			<section>
				<h2>Lorem ipsum</h2>
				<p>Lorem ipsum dolor sit amet</p>
			</section>
			<section>
				<h2>Lorem ipsum</h2>
				<p>Lorem ipsum dolor sit amet</p>
			</section>
		</article>
	</div>
	<footer>
		Footer Content
	</footer>
	<script src="js/main.js"></script>
</body>
</html>
@import "compass";
@import "partials/reset";
@import "partials/content";
html, body, div, span, applet, object, iframe, h1, h2, h3, h4, h5, h6, p, blockquote, pre, a, abbr, acronym, address, big, cite, code, del, dfn, em, img, ins, kbd, q, s, samp, small, strike, strong, sub, sup, tt, var, b, u, i, center, dl, dt, dd, ol, ul, li, fieldset, form, label, legend, table, caption, tbody, tfoot, thead, tr, th, td, article, aside, canvas, details, embed, figure, figcaption, footer, header, hgroup, menu, nav, output, ruby, section, summary, time, mark, audio, video {
	margin: 0;
	padding: 0;
	border: 0;
	font-size: 100%;
	font: inherit;
	vertical-align: baseline;
}
article, aside, details, figcaption, figure, footer, header, hgroup, menu, nav, section {
	display: block;
}
body {
	line-height: 1;
}
ol, ul {
	list-style: none;
}
blockquote, q {
	quotes: none;
}
blockquote:before, blockquote:after, q:before, q:after {
	content: '';
	content: none;
}
table {
	border-collapse: collapse;
	border-spacing: 0;
}
$color_001: #000000; // Black
$color_002: #FFFFFF; // White
$color_003: #FF0000; // Red

body {
	color: $color_002;
	background-color: $color_001;
}

Okay… now lets go to Jenkins. Here we create a new “Freestyle Job” and configure the Source-Code-Management and insert the following command into “Execute Shell”

rm -fr ${WORKSPACE}/css
compass compile

Thats it… after running the build into the workspace you should see the folder “css” and a file “style.css” the content should look like:

html,body,div,span,applet,object,iframe,h1,h2,h3,h4,h5,h6,p,blockquote,pre,a,abbr,acronym,address,big,cite,code,del,dfn,em,img,ins,kbd,q,s,samp,small,strike,strong,sub,sup,tt,var,b,u,i,center,dl,dt,dd,ol,ul,li,fieldset,form,label,legend,table,caption,tbody,tfoot,thead,tr,th,td,article,aside,canvas,details,embed,figure,figcaption,footer,header,hgroup,menu,nav,output,ruby,section,summary,time,mark,audio,video{margin:0;padding:0;border:0;font-size:100%;font:inherit;vertical-align:baseline}article,aside,details,figcaption,figure,footer,header,hgroup,menu,nav,section{display:block}body{line-height:1}ol,ul{list-style:none}blockquote,q{quotes:none}blockquote:before,blockquote:after,q:before,q:after{content:'';content:none}table{border-collapse:collapse;border-spacing:0}body{color:#fff;background-color:#000}