Command-line fake data generator

In my search for a command-line fake data generator I’ve found phony. What can I say, the tool does exactly what it should! After installation, you no longer need to leave the terminal.

Installation

# install go and git (Debian 8)
$ apt-get install -y golang git

# set GOPATH environment variable for workspace
$ mkdir ~/.go
$ echo "GOPATH=$HOME/.go" >> ~/.bashrc
$ echo "export GOPATH" >> ~/.bashrc
$ echo "PATH=\$PATH:\$GOPATH/bin" >> ~/.bashrc
$ source ~/.bashrc

# install phony
$ go get github.com/yields/phony

# verfiy installation
$ phony --version

Usage

# show help
$ phony --help

# list phony generators
$ phony --list

# generate 10 e-mails
$ echo '{{email}}' | phony --max 10

# generate 5 users (first name, last name)
$ echo 'User: {{name.first}} {{name.last}}' | phony --max 5

There is more! Look at the examples!

Record and share terminal sessions

Sometimes it is so boring to tell other software testers what to do … and nobody read documentations. Here now a easy solution! Just record and share your terminal sessions.

Installation

# install Python3 with pip (on Debian 8)
$ apt-get install -y python3 python3-pip

# verify installation
$ python3 --version && pip3 --version

# install asciinema
$ apt-get install -y asciinema

# verify installation
$ asciinema --version

Note: read the documentation of asciinema for other OS!

Usage

# start recording
$ asciinema rec

# do some great stuff on new shell instance
...

# stop recording (Ctrl-D)
$ exit

# open/send URL

Tip: Sensitive data should be shared directly (via JSON file)!

# record to local file
$ asciinema rec record.json

# distribute in any way
...

# play local record
$ asciinema play record.json

Automate Bash testing with Bats

With Bats (Bash Automated Testing System) it is easy to automate Bash and command line testing. It is an awesome open source framework written in Bash by Sam Stephenson. In combination with Jenkins you are able to use it via build.

Installation

# clone from github
$ git clone https://github.com/sstephenson/bats.git

# change directory
$ cd bats

# start installation
$ sudo ./install.sh /usr/local

Usage

# create new project
$ mkdir ~/Project/Bats && cd ~/Projects/Bats

# create Bats file
$ vim test.bats

# execute test
$ bats test.bats
...
✓ Simple check for date command
✓ Check for current user
- Test for something that does not exist (skipped: This test is skipped)
✓ Test for something that should not exist
✓ Check for individual line of output

5 tests, 0 failures, 1 skipped

# execute test with TAP output
$ bats --tap test.bats
...
1..5
ok 1 Simple check for date command
ok 2 Check for current user
ok 3 # skip (This test is skipped) Test for something that does not exist
ok 4 Test for something that should not exist
ok 5 Check for individual line of output

Example Bats file

#!/usr/bin/env bats

@test "Simple check for date command" {
  date
}

@test "Check for current user" {
  result="$(whoami)"
  [ "$result" == "lupin" ]
}

@test "Test for something that does not exist" {
  skip "This test is skipped"
  ls /test/no/test
}

@test "Test for something that should not exist" {
  run ls /test/no
  [ "$status" -eq 1 ]
}

@test "Check for individual line of output" {
  run ping -c 1 google.com
  [ "$status" -eq 0 ]
  [ "${lines[3]}" = "1 packets transmitted, 1 packets received, 0.0% packet loss" ]
}

Note: There is much more! Read documentation and have a look on projects which are using it!

Docker Audit

This tutorial shows software testers some simple examples for Docker audit. Here now we will make some audits on Docker environment and Dockerfiles.

Docker environment audit

# check Docker environment with docker-bench-security
$ docker run -it --net host --pid host --cap-add audit_control \
    -v /var/lib:/var/lib \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /usr/lib/systemd:/usr/lib/systemd \
    -v /etc :/etc --label docker_bench_security \
    docker/docker-bench-security

Note: 1st the space after /etc is only because of security settings from my provider! 2nd create os specific docker-bench-security (example CentOS)

Dockerfile audit

# install on RedHat, CentOS, Fedora ...
$ yum install epel-release && yum install lynis

# install on Debian, Ubuntu ...
$ apt-get install lynis

# Suse
$ zypper install lynis

# install via Homebrew
$ brew install lynis

# audit Dockerfile
$ lynis audit dockerfile Dockerfile

# check log file
$ cat /var/log/lynis.log
$ cat /var/log/lynis-report.dat

Lint Dockerfile with Haskell Dockerfile Linter

# simply run Container again Dockerfile
$ docker run --rm -i lukasmartinelli/hadolint < Dockerfile

SSH jump host example with vagrant

This time shows the tutorial two topics in one. The first topic is: “How an easy to configure SSH host jump”. The 2nd topic is: “Provisioning examples for Vagrant”.

Precondition

  • Vagrant installed
  • VirtualBox installed

File content

---
- name: jump_host
  box: centos/7
  ip: 192.168.100.10
  cpus: 1
  memory: 1024
- name: host_a
  box: centos/7
  ip: 192.168.100.20
  cpus: 1
  memory: 1024
- name: host_b
  box: centos/7
  ip: 192.168.100.30
  cpus: 1
  memory: 1024
Host *
  Compression yes
  AddressFamily inet
  Protocol 2
  ServerAliveInterval 60
  ServerAliveCountMax 30

Host jump_host
  HostName 192.168.x.x
  User testuser

Host host_*
  ProxyCommand ssh jump_host -W %h:%p
  User testuser
# -*- mode: ruby -*-

require 'yaml'

machines = YAML.load_file('machines.yml')
API_VERSION = "2"
KEY_LOCATION = "~/.ssh/id_rsa.pub"

Vagrant.require_version ">= 1.8.1"
Vagrant.configure(API_VERSION) do |config|

  # loop
  machines.each do |machines|
    # vm setup + provisioning
    config.vm.define machines["name"] do |machine|
      machine.vm.box = machines["box"]
      machine.vm.synced_folder ".", "/vagrant", disabled: true
      machine.vm.network "private_network", ip: machines["ip"]

      machine.vm.provider :virtualbox do |vb|
        vb.name = machines["name"]
        vb.cpus = machines["cpus"]
        vb.memory = machines["memory"]
        vb.gui = false
      end

      # provisioning: only jump_host
      if machines["name"] == 'jump_host'
        # prompt for interface
        machine.vm.network "public_network"

        machine.vm.provision "shell", inline: <<-SHELL
          cd /etc && sudo sed -i '/#AllowTcpForwarding yes/c\AllowTcpForwarding yes' ssh/sshd_config
          sudo systemctl restart sshd.service
          cd /etc && sudo grep -q 'host_a' hosts || echo '192.168.100.20  host_a' >> hosts
          cd /etc && sudo grep -q 'host_b' hosts || echo '192.168.100.30  host_b' >> hosts
        SHELL
      end

      # provisioning: all
      machine.vm.provision "file", source: KEY_LOCATION, destination: "/tmp/pub.key"

      machine.vm.provision "shell", inline: <<-SHELL
        sudo useradd testuser
        sudo mkdir /home/testuser/.ssh
        sudo touch /home/testuser/.ssh/authorized_keys
        sudo cat /tmp/pub.key > /home/testuser/.ssh/authorized_keys
        sudo rm -f /tmp/pub.key
        sudo chmod 0700 /home/testuser/.ssh
        sudo chmod 0644 /home/testuser/.ssh/authorized_keys
        sudo chown -R testuser:testuser /home/testuser/.ssh
      SHELL
    end
  end
end

Note: Replace values for HostName “192.168.x.x” by real IP from jump_host and KEY_LOCATION “~/.ssh/id_rsa.pub”!

Steps

# create and goto project
$ mkdir ~/Projects/JumpHostExample && cd ~/Projects/JumpHostExample

# create yaml file
$ vim machines.yml

# create Vagrantfile
$ vim Vagrantfile

# start Vagrant
$ vagrant up

# get IP from jump_host (for config)
$ ssh jump_host -c "ip addr show enp0s9"

# create or edit ssh config
$ vim ~/.ssh/config

# ssh into hosts via jump_host
$ ssh host_a
$ ssh host_b

Minimal CentOS 7 base box with Packer

The title says it, … this tutorial is about Packer, CentOS 7 and Vagrant. After that, you should be able to integrate the creation of Vagrant base boxes into your Build-server. There is on small exception to other – the VirtualBox Guest Additions will be provided via PlugIn! Because other users could may have different versions.

Preconditions

Project structure

$ tree
.
├── Makefile
├── packer.json
├── src
│   ├── Vagrantfile.tpl
│   └── ks.cfg
└── target

File contents

CURRENT_DIR := $(shell pwd)

.PHONY: clean

help:
	@echo "Run make with:"
	@echo " > validate       ...to run packer validation"
	@echo " > build          ...to start packer build"
	@echo " > up             ...to start vagrant"
	@echo " > reload         ...to reload vagrant"
	@echo " > ssh            ...to ssh into vm"
	@echo " > clean          ...to cleanup for next build"

validate:
	packer validate $(CURRENT_DIR)/packer.json

build:
	packer build $(CURRENT_DIR)/packer.json
	cp $(CURRENT_DIR)/src/Vagrantfile.tpl $(CURRENT_DIR)/target/Vagrantfile

up:
	vagrant box add packer/centos7 $(CURRENT_DIR)/target/virtualbox-CentOS-7.box
	cd $(CURRENT_DIR)/target && vagrant up

reload:
	cd $(CURRENT_DIR)/target && vagrant reload

ssh:
	cd $(CURRENT_DIR)/target && vagrant ssh

clean:
	cd $(CURRENT_DIR)/target && vagrant halt
	cd $(CURRENT_DIR)/target && vagrant destroy -f
	rm -fr $(CURRENT_DIR)/builds/
	rm -fr $(CURRENT_DIR)/target/* $(CURRENT_DIR)/target/.* 2> /dev/null
	vagrant box remove packer/centos7
{
  "variables": {
    "file": "http://linuxsoft.cern.ch/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1511.iso",
    "checksum": "88c0437f0a14c6e2c94426df9d43cd67",
    "type": "md5",
    "non_gui": "false"
  },
  "builders": [
    {
      "type": "virtualbox-iso",
      "iso_url": "{{ user `file` }}",
      "iso_checksum": "{{ user `checksum` }}",
      "iso_checksum_type": "md5",
      "headless": "{{ user `non_gui` }}",
      "output_directory": "builds",
      "vm_name": "CentOS7_to_Vagrant",
      "guest_os_type": "RedHat_64",
      "disk_size": "10240",
      "vboxmanage": [
        ["modifyvm", "{{.Name}}", "--memory", "2048"],
        ["modifyvm", "{{.Name}}", "--cpus", "2"],
        ["modifyvm", "{{.Name}}", "--audio", "none"],
        ["modifyvm", "{{.Name}}", "--usb", "off"]
      ],
      "http_directory": "src",
      "boot_wait": "5s",
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
      ],
      "ssh_username": "vagrant",
      "ssh_password": "vagrant",
      "ssh_port": 22,
      "ssh_wait_timeout": "600s",
      "guest_additions_path": "disable",
      "shutdown_command": "sudo shutdown -h 0"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "sudo yum update -y",
        "sudo rm -rf /tmp/*",
        "sudo rm -f /var/log/wtmp /var/log/btmp ",
        "sudo yum clean all",
        "sudo rm -rf /var/cache/* /usr/share/doc/*",
        "rm -f .bash_history",
        "history -c"
      ]
    }
  ],
  "post-processors": [
    {
      "type": "vagrant",
      "keep_input_artifact": false,
      "compression_level": 9,
      "output": "target/{{.Provider}}-CentOS-7.box"
    }
  ]
}
install
cdrom

lang en_US.UTF-8
keyboard us
timezone UTC

network --bootproto=dhcp
firewall --disabled

rootpw --plaintext packer
user --name=vagrant --password=vagrant
auth --enableshadow --passalgo=sha512 --kickstart
selinux --permissive

text
skipx

clearpart --all --initlabel
zerombr
autopart
bootloader --location=mbr

firstboot --disable
reboot

%packages --instLangs=en_US.utf8 --nobase --ignoremissing --excludedocs
@^minimal
@core

-aic94xx-firmware
-atmel-firmware
-b43-openfwwf
-bfa-firmware
-ipw2100-firmware
-ipw2200-firmware
-ivtv-firmware
-iwl100-firmware
-iwl105-firmware
-iwl135-firmware
-iwl1000-firmware
-iwl2000-firmware
-iwl2030-firmware
-iwl3160-firmware
-iwl3945-firmware
-iwl4965-firmware
-iwl5000-firmware
-iwl5150-firmware
-iwl6000-firmware
-iwl6000g2a-firmware
-iwl6000g2b-firmware
-iwl6050-firmware
-iwl7260-firmware
-libertas-usb8388-firmware
-ql2100-firmware
-ql2200-firmware
-ql23xx-firmware
-ql2400-firmware
-ql2500-firmware
-rt61pci-firmware
-rt73usb-firmware
-xorg-x11-drv-ati-firmware
-zd1211-firmware
%end

%post --log=/root/ks.log
SEE NEXT PICTURE!!!! The security settings of my provider does not allow this content!
%end

ks content

# -*- mode: ruby -*-

Vagrant.require_version ">= 1.8.1"

Vagrant.configure("2") do |config|

  config.vm.box = "packer/centos7"
  config.vm.box_url = "target/virtualbox-CentOS-7.box"
  config.vm.synced_folder ".", "/vagrant", disabled: true

  config.vm.provider "virtualbox" do |vb|
    vb.name = "CentOS-7"
    vb.cpus = "2"
    vb.memory = "2048"
    vb.gui = false
  end

end

Usage

# run packer build (via make)
$ make build

# run vagrant up (via make)
$ make run

# run vagrant reload (via make)
$ make reload

# run vagrant ssh (via make)
$ make ssh

# destroy everything (via make)
$ make clean

Create simple CentOS 7 Virtualbox with Packer

As a software tester you need many virtual machines, the creating can be very time consuming. Of course tools like Vagrant helps a lot but the creation for BaseBoxes starts most with installation from ISO`s. Exact here helps Packer! This tutorial shows an example for CentOS7 – VirtualBox.

Preconditions

Preparation

1st you need to install Packer. The following example shows one way that works well with Mac OS X (El Capitan).

# change into Downloads
$ cd ~/Downloads/

# download packer archive (Mac OS X)
$ curl -O https://releases.hashicorp.com/packer/0.10.1/packer_0.10.1_darwin_amd64.zip

# unzip packer archive
$ unzip packer_0.10.1_darwin_amd64.zip

# move packer binary
$ sudo mv packer /usr/local/bin

# check packer version
$ packer --version

Other OS? Take a look here.

Instructions

# create new project
$ mkdir ~/Projects/PackerExample && cd ~/Projects/PackerExample

# create kickstart directory and configuration
$ mkdir ~/Projects/PackerExample/http && touch ~/Projects/PackerExample/http/ks.cfg

# create new Packer JSON file
$ touch ~/Projects/PackerExample/example.json

# how it looks (before running)
$ tree .
.
├── example.json
└── http
    └── ks.cfg
{
  "variables": {
    "iso": "http://linuxsoft.cern.ch/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1511.iso",
    "checksum": "88c0437f0a14c6e2c94426df9d43cd67"
  },
  "builders": [
    {
      "type": "virtualbox-iso",
      "iso_url": "{{ user `iso` }}",
      "iso_checksum": "{{ user `checksum` }}",
      "iso_checksum_type": "md5",
      "vm_name": "MyCentOS7",
      "guest_os_type": "RedHat_64",
      "ssh_username": "root",
      "ssh_password": "packer",
      "ssh_port": 22,
      "ssh_wait_timeout": "600s",
      "vboxmanage": [
        ["modifyvm", "{{.Name}}", "--memory", "2048"],
        ["modifyvm", "{{.Name}}", "--cpus", "2"],
        ["modifyvm", "{{.Name}}", "--audio", "none"]
      ],
      "disk_size": "10240",
      "http_directory": "http",
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
      ],
      "shutdown_command": "/sbin/halt -p"
    }
  ]
}

More about Packer – VirtualBox? Take a look here.

install
cdrom
lang en_US.UTF-8
keyboard us
timezone UTC
network --bootproto=dhcp
rootpw --plaintext packer
user --name=frank --password=Test123
auth --enableshadow --passalgo=sha512 --kickstart
firewall --disabled
selinux --permissive
bootloader --location=mbr

text
skipx
zerombr

clearpart --all --initlabel
autopart

firstboot --disable
reboot

%packages --instLangs=en_US.utf8 --nobase --ignoremissing --excludedocs
@core
%end

%post --log=/root/ks.log
yum -y update
%end

More about CentOS 7 – Kickstart? Take a look here.

Validation and Build

# validate JSON
$ packer validate example.json

# run the build
$ packer build example.json

Result

$ tree .
.
├── example.json
├── http
│   └── ks.cfg
├── output-virtualbox-iso
│   ├── MyCentOS7-disk1.vmdk
│   └── MyCentOS7.ovf
└── packer_cache
    └── 4bbec2cca90f761e144becb1a24c2914eddd21d06292d6dfb415beb51ef9e69f.iso

Visualization of package dependencies

Documentation takes time – sometimes a lot of time. Here a few examples how to create dependencies pictures with Graphviz via command line. These commands can then be easily transferred to a build-process to save your time.

Mac OS X

# install Graphviz on Mac OS X
$ curl -O http://www.graphviz.org/pub/graphviz/stable/macos/mountainlion/graphviz-2.36.0.pkg
$ open graphviz-2.36.0.pkg

# check installation
$ dot -V
dot - graphviz version 2.36.0 (20140111.2315)

# clone PureDarwin
$ git clone https://github.com/PureDarwin/PureDarwin.git

# change directory
$ cd PureDarwin/scripts/

# create graph for non-installed mtr
$ sudo ./pd_portviz mtr

CentOS 7

# install Graphviz on CentOS 7
$ yum install -y graphviz

# check installation
$ dot -V
dot - graphviz version 2.30.1 (20150306.0020)

# install rpmdep
$ yum install -y epel-release && yum install -y rpmorphan

# create graph for installed which
$ rpmdep -dot which.dot which

Debian 8

# install Graphviz on Debian 8
$ apt-get install -y graphviz

# check installation
$ dot -V
dot - graphviz version 2.38.0 (20140413.2041)

# install debtree
$ apt-get install -y debtree

# create graph for non-installed make
$ debtree --with-suggests make > make.dot

Example graph for mtr on Mac OS X

# convert .dot into png
$ dot -Tpng mtr.dot -o mtr.png

mtr dependencies