Hashicorp Vault SSH OTP

With Vault’s SSH secret engine you can provide an secure authentication and authorization for SSH. With the One-Time SSH Password (OTP) you don’t need to manage keys anymore. The client requests the credentials from the Vault service and (if authorized) can connect to target service(s). Vault will take care that the OTP can be used only once and the access is logged. This tutorial will provide needed steps on a simple Docker infrastructure. Attention, in that tutorial Vault and Vault-SSH-Helper are running in Development Mode – don’t do that in production!

Conditions

Docker installed (latest CE)

Vault server

Let’s start and prepare the vault service.

# run vault-service container (local)
$ docker run -ti --name vault-service bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh sshpass

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault.zip && rm /tmp/vault.zip

# move binary
$ mv /tmp/vault /usr/local/bin/

# start vault (development mode)
$ vault server -dev -dev-listen-address='0.0.0.0:8200'

Don’t stop or close terminal session! Open new terminal. Note: The IP’s I use in this tutorial may be different to yours.

# get IP of container (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' vault-service
...
172.17.0.2
...

# run commands on container (local)
$ docker exec -ti vault-service /bin/bash

# set environment variable
$ export VAULT_ADDR='http://0.0.0.0:8200'

# enable ssh secret engine
$ vault secrets enable ssh

# create new vault role
$ vault write ssh/roles/otp_key_role key_type=otp default_user=root cidr_list=0.0.0.0/0

Target server

Now we create and configure the target service.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

# run target-service container (local)
$ docker run -ti --name target-service bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh vim

# download vault-ssh-helper
$ curl -C - -k https://releases.hashicorp.com/vault-ssh-helper/0.1.4/vault-ssh-helper_0.1.4_linux_amd64.zip -o /tmp/vault-ssh-helper.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault-ssh-helper.zip && rm /tmp/vault-ssh-helper.zip

# move binary
$ mv /tmp/vault-ssh-helper /usr/local/bin/

# create directory
$ mkdir /etc /vault-ssh-helper.d

# add content to file
$ cat > /etc /vault-ssh-helper.d/config.hcl << EOL
vault_addr = "http://172.17.0.2:8200"
ssh_mount_point = "ssh"
ca_cert = "/etc /vault-ssh-helper.d/vault.crt"
tls_skip_verify = false
allowed_roles = "*"
EOL

# verify config (optional)
$ vault-ssh-helper -dev -verify-only -config=/etc /vault-ssh-helper.d/config.hcl

Pam SSHD configuration (on target server)

# modify pam sshd configuration
$ vim /etc /pam.d/sshd
...
#@include common-auth
auth requisite pam_exec.so quiet expose_authtok log=/tmp/vaultssh.log /usr/local/bin/vault-ssh-helper -dev -config=/etc /vault-ssh-helper.d/config.hcl
auth optional pam_unix.so not_set_pass use_first_pass nodelay
...

SSHD configuration (on target server)

# modify sshd_config
$ vim /etc /ssh/sshd_config
...
ChallengeResponseAuthentication yes
UsePAM yes
PasswordAuthentication no
PermitRootLogin yes
...

# start SSHD
$ /etc /init.d/ssh start

# echo some content into file (optional)
$ echo 'Hello from target-service' > /tmp/target-service

Client server

Last container is for simulating a client.

# run some-client container (local)
$ docker run -ti --name some-client bitnami/minideb /bin/bash

# install packages
$ apt-get update && apt-get install -y ntp curl unzip ssh sshpass

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# unzip and delete archive
$ unzip -d /tmp/ /tmp/vault.zip && rm /tmp/vault.zip

# move binary
$ mv /tmp/vault /usr/local/bin/

# set environment variable
$ export VAULT_ADDR='http://172.17.0.2:8200'

# authenticate as root (root token)
$ vault auth <root token>

Usage

Most work is already done. Now we use the demo environment.

# get container IP of target-service (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' target-service
...
172.17.0.3
...

# get container IP of some-client (local)
$ docker inspect -f '{{ .NetworkSettings.IPAddress }}' some-client
...
172.17.0.4
...

# create an OTP credential (vault-service)
$ vault write ssh/creds/otp_key_role ip=172.17.0.3
$ vault write ssh/creds/otp_key_role ip=172.17.0.4

Note: Because of the security settings of my provider, spaces are after “root”. Please delete it after copy/paste.

# connect via vault SSH (vault-service)
$ vault ssh -role otp_key_role -mode otp -strict-host-key-checking=no root @172.17.0.3

# connect via vault SSH (some-client)
$ vault ssh -role otp_key_role -mode otp -strict-host-key-checking=no root @172.17.0.3

# read content of file (via SSH connections)
$ cat /tmp/target-service

# tail logfile (target-service)
$ tail -f /tmp/vaultssh.log

– Softwaretester –

knowledge is power