Multiple hosts provisioning with Vagrant, Ansible and virtualenv

In this tutorial we use Ansible (installed in virtualenv) and Vagrant. Furthermore, we have different machines (Debian, CentOS). For all hosts we want to have Provisioning on startup and via command.

Precondition

Folder structure

.
├── Makefile
├── Vagrantfile
├── playbook.yml
├── requirements.txt
└── roles
    └── common
        └── tasks
            └── main.yml

Files

ansible
ansible-lint
VAGRANTFILE_API_VERSION = "2"
 
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # CentOS 7 VM
  config.vm.define "centos" do |centos|
    centos.vm.box = "lupin/centos7"
    centos.vm.network :forwarded_port, guest: 22, host: 2221, id: 'ssh'
    centos.vm.provider "virtualbox" do |vb|
       vb.name = "CentOS7-Vagrant-Ansible"
    end
    centos.vm.provision "ansible" do |ansible|
        # ansible.verbose = "v"
        ansible.playbook = "playbook.yml"
    end
  end

  # Debian 8 VM
  config.vm.define "debian" do |debian|
    debian.vm.box = "lupin/debian8"
    debian.vm.network :forwarded_port, guest: 22, host: 2222, id: 'ssh'
    debian.vm.provider "virtualbox" do |vb|
       vb.name = "Debian-Vagrant-Ansible"
    end
    debian.vm.provision "ansible" do |ansible|
        # ansible.verbose = "v"
        ansible.playbook = "playbook.yml"
    end
  end

end
---
- hosts: all
  become: yes
  gather_facts: yes
  roles:
    - common
---
- debug: msg="System {{ ansible_distribution }}"
ENV_DIR = env
CURRENT_DIR := $(shell pwd)
INTERPRETER = $(CURRENT_DIR)/$(ENV_DIR)/bin/
PATH := ${PATH}:$(INTERPRETER)

help:
	@echo "Run make <target> with:"
	@echo " > env           : create virtualenv on folder $(ENV_DIR)"
	@echo " > deps          : install dependentcies"
	@echo " > cleanenv      : delete virtualenv"
	@echo " > start         : run vagrant up"
	@echo " > provisioning  : start ansible provisioning"
	@echo " > kill          : run vagrant destroy"

debug:
	@echo " > ansible location is     : $(INTERPRETER)"
	@echo " > environment variable is : $(PATH)"
	vagrant status

env:
	virtualenv $(ENV_DIR) && \
	. $(ENV_DIR)/bin/activate && \
	make deps

deps:
	$(ENV_DIR)/bin/pip install -r requirements.txt

cleanenv:
	rm -fr $(ENV_DIR)

start:
	vagrant up

provisioning:
	vagrant provision

kill:
	vagrant destroy -f

Usage

# create environment
$ make env

# start vagrant (create VM`s and run provisioning)
$ make start

# run provisioning (on started VM`s)
$ make provisioning

# stop vagrant (delete VM`s)
$ make kill

# delete environment
$ make cleanenv

Hint

Check out the by Vagrant generated inventory file!

$ cat .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory

PyCharm, Vagrant and Ansible

This tutorial is about the interaction of PyCharm (Community Edition), Vagrant and Ansible. I want to show how you can simplify your daily work.

Preconditions

The disclosures in the brackets are my current versions. Mac OS X user need to have Command Line Tools installed!

Folder and file structure

.
├── Makefile
├── Vagrantfile
├── inventory
├── playbook.yml
└── roles
    └── common
        └── tasks
            └── main.yml

File contents

help:
	@echo "Run make <target> with:"
	@echo " > start         : to create vm via vagrant"
	@echo " > provisioning  : to start ansible on vm"
	@echo " > kill          : to stop and destroy vm"

start:
	vagrant up

provisioning:
	ansible-playbook -i inventory playbook.yml

kill:
	vagrant destroy -f
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "demo/centos7"
  config.vm.provider "virtualbox" do |vb|
     vb.name = "Vagrant-Ansible"
  end
  config.vm.provision "ansible" do |ansible|
      # ansible.verbose = "v"
      ansible.playbook = "playbook.yml"
  end
end
[vagrant-example]
127.0.0.1 ansible_ssh_user=vagrant ansible_ssh_port=2222 ansible_ssh_private_key_file=.vagrant/machines/default/virtualbox/private_key
---
- hosts: all
  become: yes
  gather_facts: yes
  roles:
    - common
---
- name: upgrade all packages via yum
  yum: name=* state=latest
  when: (ansible_distribution == 'CentOS') or
        (ansible_distribution == 'Red Hat Enterprise Linux')
  tags:
    - common

- name: upgrade all packages via apt
  apt: upgrade=dist
  when: (ansible_distribution == 'Debian') or
        (ansible_distribution == 'Ubuntu')
  tags:
    - common

Little hint

If you do not know the path for ansible_ssh_private_key_file, just type $ vagrant ssh-config!

PyCharm – External Tools

In the last step we configure the PyCharm (External Tools). We do this for every command from Makefile exept help.

PyCharm-ExternalTool Configuration

PyCharm Make Commands

Create private vagrant box repository

For various reasons it may happen that the public Vagrant box repository should not be used. But Vagrant boxes should be available on the internal network. In addition, these boxes are to be versioned. With a few steps, this can be realized by Nginx.

Preparation

Nginx

# install epel-release
$ yum install -y epel-release && yum update -y

# install nginx, and additional tools
$ yum install -y nginx vim tree

# configure nginx to be automatically started at boot 
$ systemctl enable nginx

# start nginx
$ systemctl start nginx

# create new folders
$ mkdir -p /usr/share/nginx/html/devops/vagrant/boxes

# set access rights
$ chmod 755 -R /usr/share/nginx/html/devops/

# create JSON file
$ touch /usr/share/nginx/html/devops/vagrant/centos.json

# show current state
$ tree /usr/share/nginx/html/devops/
/usr/share/nginx/html/devops/
└── vagrant
    ├── boxes
    │ └── CentOS7.box
    └── centos.json

# get sha1 checksum from box
$ sha1sum /usr/share/nginx/html/devops/vagrant/boxes/CentOS7.box

# edit JSON file
$ vim /usr/share/nginx/html/devops/vagrant/centos.json

# edit nginx conf
$ vim /etc /nginx/nginx.conf

# check nginx config
$ nginx -t

# restart nginx
$ systemctl restart nginx

# enter permissive mode
$ setenforce 0
{
    "name": "demo/centos7",
    "description": "This box contains CentOS 7 64-bit.",
    "versions": [{
        "version": "0.1.0",
        "providers": [{
                "name": "virtualbox",
                "url": "http://<target>/devops/vagrant/boxes/CentOS7.box",
                "checksum_type": "sha1",
                "checksum": "99e6d7fc44cccabdfc6ed9ce178ca65fd9dcbac8"
        }]
    }]
}
...

# resolve vagrant json file(s)
location ~ ^/devops/vagrant/$ {
    index $1.json;
    try_files $uri $uri/ $1.json =404;
    autoindex off;
}
# enable auto indexing for boxes
location ~ ^/devops/vagrant/boxes/$ {
    try_files $uri $uri/ =404;
    autoindex on;
    autoindex_exact_size on;
    autoindex_localtime on;
}
# serve json with specific header
    location ~ \.json$ {
    add_header Content-Type application/json;
}
# serve box(s) with specific header
location ~ \.box$ {
    add_header Content-Type application/octet-stream;
}

...

Client

On your browser check following URL`s before proceeding: http://<target>/devops/vagrant/centos.json and http://<target>/devops/vagrant/boxes/

# create project directory
$ mkdir ~/tutorial && cd ~/tutorial

# add base box repository
$ vagrant box add demo/centos7 http://<target>/devops/vagrant/centos.json

# list all boxes
$ vagrant box list
demo/centos7  (virtualbox, 0.1.0)

# create Vagrant project
$ vagrant init demo/centos7

Nginx (Part 2)

# edit JSON file
$ vim /usr/share/nginx/html/devops/vagrant/centos.json

Attention! I use for this demonstration the same box!

{
    "name": "demo/centos7",
    "description": "This box contains CentOS 7 64-bit.",
    "versions": [{
        "version": "0.1.0",
        "providers": [{
                "name": "virtualbox",
                "url": "http://<target>/devops/vagrant/boxes/CentOS7.box",
                "checksum_type": "sha1",
                "checksum": "99e6d7fc44cccabdfc6ed9ce178ca65fd9dcbac8"
        }]
    },{
        "version": "0.1.1",
        "providers": [{
                "name": "virtualbox",
                "url": "http://<target>/devops/vagrant/boxes/CentOS7.box",
                "checksum_type": "sha1",
                "checksum": "99e6d7fc44cccabdfc6ed9ce178ca65fd9dcbac8"
        }]
    }]
}

Client (Part 2)

# check box version
$ vagrant box outdated
A newer version of the box 'demo/centos7' is available! You currently
have version '0.1.0'. The latest is version '0.1.1'. Run
`vagrant box update` to update.

You can add now more boxes with different JSON files!

Reaver, Wash and CentOS 7

In part 3, I show how to install Reaver/Wash on CentOS 7.

Preparation

Installation

# download reaver and wash
$ wget https://reaver-wps.googlecode.com/files/reaver-1.4.tar.gz

# unzip
$ tar -zxvf reaver-1.4.tar.gz

# install reaver and wash
$ cd /reaver-1.4/src
$ ./configure
$ make install

# optional read docs
$ cat /reaver-1.4/docs/README.REAVER
$ cat /reaver-1.4/docs/README.WASH

Usage

# kill interfering processes
$ airmon-ng check kill

# set interface into monitor mode (my interface is wlp0s11u1)
$ airmon-ng start wlp0s11u1

# find WPS routers via wash
$ wash -I wlp0s11u1mon

# start reaver running
$ reaver -i wlp0s11u1mon -b <ESSID> -t 2 -vv