Vagrant and Vault

I was a little surprised why there is no Vagrant plug-in for Vault. Then I thought no matter, because the Vagrantfile is actually a Ruby script. Let me try it. I have to say right away that I’m not a Ruby developer! But here is my solution which has brought me to the goal.

Prerequisite

  • latest Vault installed (0.11.0)
  • latest Vagrant installed (2.1.3)

Prepare project and start Vault

# create new project
$ mkdir -p ~/Projects/vagrant-vault && cd ~/Projects/vagrant-vault

# create 2 empty files
$ touch vagrant.hcl Vagrantfile

# start Vault in development mode
$ vault server -dev

Here my simple vagrant policy (don’t do that in production).

path "secret/*" {
  capabilities = ["read", "list"]
}

And here is my crazy and fancy Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'net/http'
require 'uri'
require 'json'
require 'ostruct'

################ YOUR SETTINGS ####################
ROLE_ID = '99252343-090b-7fb0-aa26-f8db3f5d4f4d'
SECRET_ID = 'b212fb14-b7a4-34d3-2ce0-76fe85369434'
URL = 'http://127.0.0.1:8200/v1/'
SECRET_PATH = 'secret/data/vagrant/test'
###################################################

def getToken(url, role_id, secret_id)
  uri = URI.parse(url + 'auth/approle/login')
  request = Net::HTTP::Post.new(uri)
  request.body = JSON.dump({
    "role_id" => role_id,
    "secret_id" => secret_id
  })

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    token = result.auth.client_token
    return token
  else
    return ''
  end
end

def getSecret(url, secret_url, token)
  uri = URI.parse(url + secret_url)
  request = Net::HTTP::Get.new(uri)
  request["X-Vault-Token"] = token

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    return result
  else
    return ''
  end
end

token = getToken(URL, ROLE_ID, SECRET_ID)

unless token.to_s.strip.empty?
  result = getSecret(URL, SECRET_PATH, token)
  unless result.to_s.strip.empty?
    sec_a = result.data.data.secret_a
    sec_b = result.data.data.secret_b
  end
else
  puts 'Error - please check your settings'
  exit(1)
end

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.post_up_message = 'Secret A:' + sec_a + ' - Secret B:' + sec_b
end

Configure Vault

# set environment variables (new terminal)
$ export VAULT_ADDR='http://127.0.0.1:8200'

# check status (optional)
$ vault status

# create simple kv secret
$ vault kv put secret/vagrant/test secret_a=foo secret_b=bar

# show created secret (optional)
$ vault kv get --format yaml -field=data secret/vagrant/test

# create/import vagrant policy
$ vault policy write vagrant vagrant.hcl

# show created policy (optional)
$ vault policy read vagrant

# enable AppRole auth method
$ vault auth enable approle

# create new role
$ vault write auth/approle/role/vagrant token_num_uses=1 token_ttl=10m token_max_ttl=20m policies=vagrant

# show created role (optional)
$ vault read auth/approle/role/vagrant

# show role_id
$ vault read auth/approle/role/vagrant/role-id
...
99252343-090b-7fb0-aa26-f8db3f5d4f4d
...

# create and show secret_id
$ vault write -f auth/approle/role/vagrant/secret-id
...
b212fb14-b7a4-34d3-2ce0-76fe85369434
...

Run it

# starts and provisions the vagrant environment
$ vagrant up

😉 … it just works

Unseal Vault with PGP

In this tutorial I will show an example for unsealing Vault using GPG. We generate for two users the keys and each user will use them to unseal. For the storage we use Consul.

Conditions

Host Preparation

First we need to setup, configure and start Consul and Vault.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

# create new project
$ mkdir -p ~/Projects/VaultConsulPGP/consul-data && cd ~/Projects/VaultConsulPGP

# create private/public keys
$ openssl req -newkey rsa:4096 -nodes -keyout private_key.pem -x509 -days 365 -out public_key.pem
...
Country Name (2 letter code) []:CH
State or Province Name (full name) []:Zuerich
Locality Name (eg, city) []:Winterthur
Organization Name (eg, company) []:Softwaretester
Organizational Unit Name (eg, section) []:QA
Common Name (eg, fully qualified host name) []:demo.env
Email Address []:demo@demo.env
...

# create HCL configuration
$ touch ~/Projects/VaultConsulPGP/vault.hcl

# add hosts entry
$ echo -e "127.0.0.1 demo.env\n" >> /etc /hosts

# start consul service
$ consul agent -server -bootstrap-expect 1 -data-dir $HOME/Projects/VaultConsulPGP/consul-data -ui

# start vault service
$ vault server -config $HOME/Projects/VaultConsulPGP/vault.hcl

Do not stop and/or close any terminal sessions!

ui = true

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault"
}

listener "tcp" {
  address       = "demo.env:8200"
  tls_cert_file = "public_key.pem"
  tls_key_file  = "private_key.pem"
}

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
|____public_key.pem

Client Preparation

As I wrote – we need to simulate two users. Now to the Docker client’s…

# run client A
$ docker run -ti --name client_a --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

# run client B
docker run -ti --name client_b --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

Both client’s need similar configuration, so please execute the following steps on both containers.

# install needed packages
$ apt-get update && apt-get install -y curl unzip gnupg iputils-ping

# get host IP
$ HOST_IP=$(ping -c 1 host.docker.internal | grep "64 bytes from"|awk '{print $4}')

# add hosts entry
$ echo -e "${HOST_IP} demo.env\n" >> /etc /hosts

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# extract archive and move binary and clean up
$ unzip -d /tmp /tmp/vault.zip && mv /tmp/vault /usr/local/bin/ && rm /tmp/vault.zip

# generate GPG key (1x for each client)
$ gpg --gen-key
...
Real name: usera
...
Real name: userb
...
# don't set a passphrase!!!!

# export generated key (client 1)
$ gpg --export [UID] | base64 > /tmp/target/usera.asc

# export generated key (client 2)
$ gpg --export [UID] | base64 > /tmp/target/userb.asc

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
     |____ ...
     |____ ...
|____public_key.pem
|____usera.asc
|____userb.asc

Initialize and Unseal Vault

On the host we initialize the Vault and share unseal key’s back to the client’s.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=public_key.pem

# ensure proper location (host)
cd ~/Projects/VaultConsulPGP

# initialize vault
$ vault operator init -key-shares=2 -key-threshold=2 -pgp-keys="usera.asc,userb.asc"

Note: Save now all keys and share the correspondending <unseal keys> to the client’s!

Now our client’s can start the unseal of Vault. Even here, please execute the following steps on both containers.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=/tmp/target/public_key.pem

# decode unseal key
$ echo "<unseal key>" | base64 -d | gpg -dq

# unseal vault
$ vault operator unseal <...>

Just for information

We configured both services (Consul and Vault) with WebUI.

# open Consul in Firefox
$ open -a Firefox http://127.0.0.1:8500

# open Vault in Firefox
$ open -a Firefox https://demo.env:8200/ui

Use the “Initial Root Token” to login into Vault’s WebUI.