Do you like Vagrant? … Yes? … Then you will love Vagrant Manger! Vagrant Manager is free and available for Mac OS and Windows. But pictures say more than 1000 words…
Windows
Copyright © by Lanayo Technologies
Mac OS
Copyright © by Lanayo Technologies
Downloads are available via GitHub (Windows – Mac OS). The developers of Lanayo have done a very good job here!
This time shows the tutorial two topics in one. The first topic is: “How an easy to configure SSH host jump”. The 2nd topic is: “Provisioning examples for Vagrant”.
Precondition
File content
--- - name: jump_host box: centos/7 ip: 192.168.100.10 cpus: 1 memory: 1024 - name: host_a box: centos/7 ip: 192.168.100.20 cpus: 1 memory: 1024 - name: host_b box: centos/7 ip: 192.168.100.30 cpus: 1 memory: 1024
Host * Compression yes AddressFamily inet Protocol 2 ServerAliveInterval 60 ServerAliveCountMax 30 Host jump_host HostName 192.168.x.x User testuser Host host_* ProxyCommand ssh jump_host -W %h:%p User testuser
# -*- mode: ruby -*-
require 'yaml'
machines = YAML.load_file('machines.yml')
API_VERSION = "2"
KEY_LOCATION = "~/.ssh/id_rsa.pub"
Vagrant.require_version ">= 1.8.1"
Vagrant.configure(API_VERSION) do |config|
# loop
machines.each do |machines|
# vm setup + provisioning
config.vm.define machines["name"] do |machine|
machine.vm.box = machines["box"]
machine.vm.synced_folder ".", "/vagrant", disabled: true
machine.vm.network "private_network", ip: machines["ip"]
machine.vm.provider :virtualbox do |vb|
vb.name = machines["name"]
vb.cpus = machines["cpus"]
vb.memory = machines["memory"]
vb.gui = false
end
# provisioning: only jump_host
if machines["name"] == 'jump_host'
# prompt for interface
machine.vm.network "public_network"
machine.vm.provision "shell", inline: <<-SHELL
cd /etc && sudo sed -i '/#AllowTcpForwarding yes/c\AllowTcpForwarding yes' ssh/sshd_config
sudo systemctl restart sshd.service
cd /etc && sudo grep -q 'host_a' hosts || echo '192.168.100.20 host_a' >> hosts
cd /etc && sudo grep -q 'host_b' hosts || echo '192.168.100.30 host_b' >> hosts
SHELL
end
# provisioning: all
machine.vm.provision "file", source: KEY_LOCATION, destination: "/tmp/pub.key"
machine.vm.provision "shell", inline: <<-SHELL
sudo useradd testuser
sudo mkdir /home/testuser/.ssh
sudo touch /home/testuser/.ssh/authorized_keys
sudo cat /tmp/pub.key > /home/testuser/.ssh/authorized_keys
sudo rm -f /tmp/pub.key
sudo chmod 0700 /home/testuser/.ssh
sudo chmod 0644 /home/testuser/.ssh/authorized_keys
sudo chown -R testuser:testuser /home/testuser/.ssh
SHELL
end
end
end
Note: Replace values for HostName “192.168.x.x” by real IP from jump_host and KEY_LOCATION “~/.ssh/id_rsa.pub”!
Steps
# create and goto project $ mkdir ~/Projects/JumpHostExample && cd ~/Projects/JumpHostExample # create yaml file $ vim machines.yml # create Vagrantfile $ vim Vagrantfile # start Vagrant $ vagrant up # get IP from jump_host (for config) $ ssh jump_host -c "ip addr show enp0s9" # create or edit ssh config $ vim ~/.ssh/config # ssh into hosts via jump_host $ ssh host_a $ ssh host_b
The title says it, … this tutorial is about Packer, CentOS 7 and Vagrant. After that, you should be able to integrate the creation of Vagrant base boxes into your Build-server. There is on small exception to other – the VirtualBox Guest Additions will be provided via PlugIn! Because other users could may have different versions.
Preconditions
Project structure
$ tree . ├── Makefile ├── packer.json ├── src │ ├── Vagrantfile.tpl │ └── ks.cfg └── target
File contents
CURRENT_DIR := $(shell pwd) .PHONY: clean help: @echo "Run make with:" @echo " > validate ...to run packer validation" @echo " > build ...to start packer build" @echo " > up ...to start vagrant" @echo " > reload ...to reload vagrant" @echo " > ssh ...to ssh into vm" @echo " > clean ...to cleanup for next build" validate: packer validate $(CURRENT_DIR)/packer.json build: packer build $(CURRENT_DIR)/packer.json cp $(CURRENT_DIR)/src/Vagrantfile.tpl $(CURRENT_DIR)/target/Vagrantfile up: vagrant box add packer/centos7 $(CURRENT_DIR)/target/virtualbox-CentOS-7.box cd $(CURRENT_DIR)/target && vagrant up reload: cd $(CURRENT_DIR)/target && vagrant reload ssh: cd $(CURRENT_DIR)/target && vagrant ssh clean: cd $(CURRENT_DIR)/target && vagrant halt cd $(CURRENT_DIR)/target && vagrant destroy -f rm -fr $(CURRENT_DIR)/builds/ rm -fr $(CURRENT_DIR)/target/* $(CURRENT_DIR)/target/.* 2> /dev/null vagrant box remove packer/centos7
{
"variables": {
"file": "http://linuxsoft.cern.ch/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1511.iso",
"checksum": "88c0437f0a14c6e2c94426df9d43cd67",
"type": "md5",
"non_gui": "false"
},
"builders": [
{
"type": "virtualbox-iso",
"iso_url": "{{ user `file` }}",
"iso_checksum": "{{ user `checksum` }}",
"iso_checksum_type": "md5",
"headless": "{{ user `non_gui` }}",
"output_directory": "builds",
"vm_name": "CentOS7_to_Vagrant",
"guest_os_type": "RedHat_64",
"disk_size": "10240",
"vboxmanage": [
["modifyvm", "{{.Name}}", "--memory", "2048"],
["modifyvm", "{{.Name}}", "--cpus", "2"],
["modifyvm", "{{.Name}}", "--audio", "none"],
["modifyvm", "{{.Name}}", "--usb", "off"]
],
"http_directory": "src",
"boot_wait": "5s",
"boot_command": [
"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
],
"ssh_username": "vagrant",
"ssh_password": "vagrant",
"ssh_port": 22,
"ssh_wait_timeout": "600s",
"guest_additions_path": "disable",
"shutdown_command": "sudo shutdown -h 0"
}
],
"provisioners": [
{
"type": "shell",
"inline": [
"sudo yum update -y",
"sudo rm -rf /tmp/*",
"sudo rm -f /var/log/wtmp /var/log/btmp ",
"sudo yum clean all",
"sudo rm -rf /var/cache/* /usr/share/doc/*",
"rm -f .bash_history",
"history -c"
]
}
],
"post-processors": [
{
"type": "vagrant",
"keep_input_artifact": false,
"compression_level": 9,
"output": "target/{{.Provider}}-CentOS-7.box"
}
]
}
install cdrom lang en_US.UTF-8 keyboard us timezone UTC network --bootproto=dhcp firewall --disabled rootpw --plaintext packer user --name=vagrant --password=vagrant auth --enableshadow --passalgo=sha512 --kickstart selinux --permissive text skipx clearpart --all --initlabel zerombr autopart bootloader --location=mbr firstboot --disable reboot %packages --instLangs=en_US.utf8 --nobase --ignoremissing --excludedocs @^minimal @core -aic94xx-firmware -atmel-firmware -b43-openfwwf -bfa-firmware -ipw2100-firmware -ipw2200-firmware -ivtv-firmware -iwl100-firmware -iwl105-firmware -iwl135-firmware -iwl1000-firmware -iwl2000-firmware -iwl2030-firmware -iwl3160-firmware -iwl3945-firmware -iwl4965-firmware -iwl5000-firmware -iwl5150-firmware -iwl6000-firmware -iwl6000g2a-firmware -iwl6000g2b-firmware -iwl6050-firmware -iwl7260-firmware -libertas-usb8388-firmware -ql2100-firmware -ql2200-firmware -ql23xx-firmware -ql2400-firmware -ql2500-firmware -rt61pci-firmware -rt73usb-firmware -xorg-x11-drv-ati-firmware -zd1211-firmware %end %post --log=/root/ks.log SEE NEXT PICTURE!!!! The security settings of my provider does not allow this content! %end
# -*- mode: ruby -*-
Vagrant.require_version ">= 1.8.1"
Vagrant.configure("2") do |config|
config.vm.box = "packer/centos7"
config.vm.box_url = "target/virtualbox-CentOS-7.box"
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.provider "virtualbox" do |vb|
vb.name = "CentOS-7"
vb.cpus = "2"
vb.memory = "2048"
vb.gui = false
end
endUsage
# run packer build (via make) $ make build # run vagrant up (via make) $ make run # run vagrant reload (via make) $ make reload # run vagrant ssh (via make) $ make ssh # destroy everything (via make) $ make clean
Depending on the vagrant version and how you build/package boxes, you may know the Authentication failure after command “$ vagrant up“. Here now two simple tips, how to prevent or fix it.
Prevention
If you build on an existing box you should not allow Vagrant to automatically insert a new key pair.
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure('2') do |config|
config.ssh.insert_key = false
...
endWorkaround
If the problem already exists…
# start VM (local) $ vagrant up ... default: Warning: Authentication failure. Retrying... Timed out while waiting for the machine to boot... ... # login via password authentication (local -> VM) $ vagrant ssh # start edit authorized_keys (VM) [vagrant@acme ~]$ vim .ssh/authorized_keys # add latest key from https://github.com/mitchellh/vagrant/blob/master/keys/vagrant.pub and save # leave vm (VM -> local) $ exit # stop VM (local) $ vagrant halt
On command “$ vagrant halt” Vagrant will automatically insert a new keypair.
You may need to watch different log files on automated test runs. With log.io you can simply monitoring log files via browser! This tutorial shows how easy it is.
Preconditions
Preparation
Create new project with following structure and files.
# create new project LogIO
$ mkdir -p ~/Projects/LogIO/data
# go into new Project
$ cd ~/Projects/LogIO
# create needed files in data
$ touch data/{harvester.conf,log_server.conf,web_server.conf,log.io}
# create Vagrantfile
$ touch Vagrantfile
# show files
$ tree .
.
├── Vagrantfile
└── data
├── harvester.conf
├── log.io
├── log_server.conf
└── web_server.confFile contents
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.require_version ">= 1.8.1"
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "centos/7"
config.vm.network "public_network"
config.vm.synced_folder "./data", "/vagrant", disabled: false
config.vm.provider "virtualbox" do |vb|
vb.name = "LogIO"
vb.cpus = "2"
vb.memory = "2048"
vb.gui = false
end
config.vm.provision "shell", inline: <<-SHELL
# install needed packages
sudo yum update -y && sudo yum install -y epel-release
sudo yum install -y vim net-tools npm nodejs
sudo yum clean all
# install log.io for user <root>
sudo npm install -g log.io --user "root"
# provide custom files for user <root>
sudo rm -f /root/.log.io/*
sudo cp /vagrant/*.conf /root/.log.io/
sudo chown root:root /root/.log.io/*.conf
# provide init.d for log.io
sudo cp /vagrant/log.io /usr/local/bin/log.io
sudo chmod +x /usr/local/bin/log.io
sudo chown root:root /usr/local/bin/log.io
SHELL
end
Configure your Harvesters…
exports.config = {
nodeName: "application_server",
logStreams: {
apache: [
"/var/log/apache2/access.log",
"/var/log/apache2/error.log"
]
},
server: {
// connect to log.io server
host: '127.0.0.1',
port: 28777
}
}Configure your log server…
exports.config = {
host: '0.0.0.0',
port: 28777
}Configure your web server…
exports.config = {
host: '0.0.0.0',
port: 28778,
/*
// Enable HTTP Basic Authentication
auth: {
user: "admin",
pass: "1234"
},
*/
/*
// Enable HTTPS/SSL
ssl: {
key: '/path/to/privatekey.pem',
cert: '/path/to/certificate.pem'
},
*/
/*
// Restrict access to websocket (socket.io)
// Uses socket.io 'origins' syntax
restrictSocket: '*:*',
*/
/*
// Restrict access to http server (express)
restrictHTTP: [
"192.168.29.39",
"10.0.*"
]
*/
}
Create simple init script…
#!/bin/bash
start() {
echo "Starting log.io process..."
/usr/bin/log.io-server &
/usr/bin/log.io-harvester &
}
stop() {
echo "Stopping io-log process..."
pkill node
}
status() {
echo "Status io-log process..."
netstat -tlp | grep node
}
case "$1" in
start) start;;
stop) stop;;
status) status;;
*) echo "Usage: start|stop|status";;
esacUsage
# start VM via vagrant $ vagrant up # SSH into VM $ vagrant ssh # become root $ sudo su - # start log.io $ log.io start # get ip $ ip addr
Now open your browser with URL http://<ip>:28778
This time again some topics in one tutorial. We touch Vagrant with Shell provision (external script), Docker Remote API and something CentOS 7 configuration.
Preconditions
Preparation
If you already have a CentOS 7 with Docker, you can skip these steps. Create a folder with the following 2 files…
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.require_version ">= 1.8.1"
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.define "centos" do |centos|
centos.vm.box = "centos/7"
centos.vm.network "public_network"
centos.vm.synced_folder ".", "/vagrant", disabled: true
centos.vm.provider "virtualbox" do |vb|
vb.name = "Docker-Jenkins"
vb.cpus = "2"
vb.memory = "2048"
end
centos.vm.provision "shell", path: "provision.sh"
end
end#!/usr/bin/env bash yum update -y && yum install -y epel-release vim tee /etc /yum.repos.d/docker.repo <<-'EOF' [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/ enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg EOF yum install -y docker-engine jq systemctl enable docker systemctl start docker
Instructions
# start VM via Vagrant $ vagrant up # SSH into VM via Vagrant $ vagrant ssh # check if API is enabled (optional) $ docker -H=tcp://127.0.0.1:4243 images
Now we edit docker.service file. The path may be different!
# find docker.service file $ systemctl status docker # edit docker.service $ vim /usr/lib/systemd/system/docker.service ... [Service] ExecStart= ExecStart=/usr/bin/docker daemon -H fd:// -D -H tcp://127.0.0.1:4243 ... # reload and restart docker service $ systemctl daemon-reload && systemctl restart docker # test docker API again $ docker -H=tcp://127.0.0.1:4243 version
Note: Be careful if you use tcp://0.0.0.0:4243 !!!!
# some tests with curl and jq on Docker Remote API $ curl -X GET http://127.0.0.1:4243/version | jq . $ curl -X GET http://127.0.0.1:4243/info | jq .DriverStatus $ curl -X GET http://127.0.0.1:4243/networks | jq '.[0]'
Read the manual of jq, the power of curl and jq together is awesome.
Today a few Vagrant Plugin Recommendations.
Preconditions
Preparation
Create a simple Vagrant BaseBox (as here) and if you like use the following Vagrantfile.
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "lupin/centos7"
# config.vm.network "public_network"
config.vm.synced_folder ".", "/vagrant", disabled: true
# config.vm.synced_folder "./shared", "/vagrant"
config.vm.provider "virtualbox" do |vb|
vb.cpus = "2"
vb.memory = "2048"
end
if Vagrant.has_plugin?("vagrant-hostsupdater")
config.vm.network :private_network, ip: "10.0.0.10"
config.hostsupdater.aliases = ["example.test"]
end
config.vm.provision "shell", inline: <<-SHELL
sudo yum update -y
sudo yum install -y vim docker
sudo yum clean all
sudo systemctl enable docker
sudo systemctl start docker
sudo setenforce 0
SHELL
config.vm.provision "docker" do |d|
d.run "nginx", args: "-p 8080:80"
end
config.vm.provision "shell", inline: "sudo hostname -I"
end
1. vagrant-camera
# install camera plugin $ vagrant plugin install vagrant-camera # start VM via Vagrant $ vagrant up # do screenshot on current target folder $ vagrant camera -s . # show all pictures (Mac OS X) $ open screenshot_default_*.png
2. vagrant-vbguest
# start VM via Vagrant $ vagrant up # ssh into VM (optional) $ vagrant ssh # check current version (optional) $ VBoxControl --version ... 5.0.20r106931 # exit from VM (optional) $ exit # install vbguest plugin $ vagrant plugin install vagrant-vbguest # reload VM $ vagrant reload # ssh into VM (optional) $ vagrant ssh # list all downloaded (optional) $ ls -l /opt ... VBoxGuestAdditions-5.0.14 VBoxGuestAdditions-5.0.20
3. vagrant-hostsupdater
# install hostsupdater plugin $ vagrant plugin install vagrant-hostsupdater # check content of local hosts (optional) $ cd /etc && cat hosts # start VM $ vagrant up # or reload if already running $ vagrant reload --provision # check content of local hosts (optional) $ cd /etc && cat hosts # check status $ vagrant ssh -c 'curl -v example.test:8080' # stop VM $ vagrant halt # check content of local hosts (optional) $ cd /etc && cat hosts
Ruby`s stdlib provides an YAML module for data serialization. By using this module, we can create only one Vagrantfile which reads the configuration from YAML files. If you are thinking of build servers – so only YAML files are needed for generating complete environments.
Preconditions
Preparation
# create new directory $ mkdir ~/Projects/Tutorial && cd ~/Projects/Tutorial # create new files $ touch Vagrantfile && touch server-config.yml # list available BaseBoxes (optional) $ vagrant box list # edit Vagrantfile $ vim Vagrantfile # edit server-config.yml $ vim server-config.yml
# -*- mode: ruby -*-
require 'yaml'
servers = YAML.load_file('server-config.yml')
API_VERSION = "2"
Vagrant.configure(API_VERSION) do |config|
servers.each do |servers|
config.vm.define servers["name"] do |machine|
machine.vm.box = servers["box"]
machine.vm.network :forwarded_port, guest: 22, host: servers["ssh"], id: 'ssh'
machine.vm.provider :virtualbox do |vb|
vb.name = servers["name"]
vb.memory = servers["memory"]
vb.cpus = servers["cpus"]
end
end
end
end--- - name: box_centos7_a box: lupin/centos7 ssh: 2221 memory: 1024 cpus: 2 - name: box_centos7_b box: lupin/centos7 ssh: 2222 memory: 1024 cpus: 2
Usage
# start run $ vagrant up # check status $ vagrant status # SSH example $ vagrant ssh [name]