Vagrant and Vault

I was a little surprised why there is no Vagrant plug-in for Vault. Then I thought no matter, because the Vagrantfile is actually a Ruby script. Let me try it. I have to say right away that I’m not a Ruby developer! But here is my solution which has brought me to the goal.

Prerequisite

  • latest Vault installed (0.11.0)
  • latest Vagrant installed (2.1.3)

Prepare project and start Vault

Here my simple vagrant policy (don’t do that in production).

And here is my crazy and fancy Vagrantfile

Configure Vault

Run it

πŸ˜‰ … it just works

Start with Vault 0.10.x

HashiCorp released Vault version 0.10.x on April 2018. The 0.10.x release delivers many new features and changes (eq. K/V Secrets Engine v2, Vault Web UI, etc.). Please have a look on vault/CHANGELOG for more informations. This tiny tutorial will concentrate now on usage of Vault’s Key-Value Secrets Engine via CLI.

Preparation

Start Vault server in development mode

Do not stop the process and open new tab on terminal [COMMAND] + [t].

Create, Read, Update and Delete secrets

As you can see, there are minor changes to previous versions of Vault.

Note: The API for the Vault KV secrets engine even changed.

Okay, back to CLI and some examples which are better for automation. We will use the STDIN and a simple JSON file.

Web UI

Previously the Web UI was for Enterprise only, now it has been made open source.

Now you can use the root token to sign in.

Create a simple video test environment (Part 3)

Okay, now is time to see some command line tools to analysis videos. I selected 4 Open-Source applications (avprobe, mediainfo, mplayer, exiftool).

Specification

  • docker
  • git

Get ready for docker images

On Bitbucket I created a repository with needed Dockerfiles for fast usage. You can also choose the installation method.

mediainfo

Lets start with mediainfo. Here some information about on wikipedia.

mplayer

Second application is mplayer. Here the wikipedia link.

exiftool

Now we take a look on exiftool. Here the wikipedia article and the official documentation.

avprobe

Last but not least avprobe. Here the wikipedia article and detailed official documentation.

Compare tools by expecting specific result

I will not judge the applications against each other! But here a compare of complexity of commands and output for video duration.

Create a simple video test environment

In this series I would like to clarify the following questions. How can you test local videos and videos provided by a server? What tools are there? How could the test environment look like? … So these tutorials should provide an entry into the subject of video testing. In this part, we will provide the test environment.

Specification

additional Software

Files

Project Structure

Prepare and run your test environment

The test environment will created by Vagrant. The Vagrant Base box needs to be provided with Debian 9. Therefore you have now 2 options. You can use a Debian 9 Base box from Vagrant cloud or you create your own. To create your own Debian 9 Base box you can use my Packer Git repository.

Please ensure, that the Β Vagrantfile is properly configured (config.vm.box_url, config.vm.box) before start-up environment.

After successful start you will find some informations on start pageΒ about How-to create, modify, upload and stream videos. In second tutorial we will have a detailed look on it.

StartPage Video Test Environment

To understand the background somewhat better, take a look!

Develop some more

There are even more ways to publish video files (Media Streaming Server). You can easily expand the environment. Just customize/create the configurations and files in the “src” folder as well as the Vagrantfile. For Dynamic Adaptive Streaming over HTTP (DASH) support take a look here. This link opens the commercial software solution.

Note: You simply link the IDE with the “src” folder. To change the images and videos, just run vagant provisioner.

Download and install ffmpeg binaries on macOS

Currently official ffmpeg packages are available for Windows, Linux (Debian, Ubuntu, Fedora, RedHat) and macOS. You can download latest versions here.

Here now a solution for macOS users, if you don’t like to install many additional software on your system (static FFmpeg binaries).

Now some basic ffmpeg commands, which should work now. If you like,you can move the binary into directory “/usr/local/”.

Okay, that’s it for first tutorial.

Simple Vault introduction

Today a tiny introduction to Vault from HashiCorp. I will just show the simplest usage. But this will help to get a first idea of Vault and the features.

Requirements

Preparation

Start Vault Server

Copy Root Token value to clipboard!!!

Insomnia

Create new environment “vault” under “Manage Environments” and store here your URL as “base_url” and Root Token as “api_key”.

insomnia vault environment

Now we create 4 simple requests

insomnia requests

for all requests we add Header

insomnia header

For first URL (POST: Add new secret) we use “{{ base_url }}/secret/MyFirstSecret” and we add following body as JSON.

After send the key:value is stored inside Vault. You can modify the request (e.q. “{{ base_url }}/secret/MySecondSecret”) and send some more.

Our next request is to show all keys (GET: Get list of secret keys)Β “{{ base_url }}/secret?list=true”.Β The Preview will show similar output.

insomnia get vault keys

3rd request is to get the value from a specific key (GET: Get value of specific secret)Β “{{ base_url }}/secret/MySecret”.

insomnia get vault value

Last request is for delete (DEL: Delete specific secret)Β “{{ base_url }}/secret/MySecret”.

Tipp: if you lost the root token (Vault server is running) you can find the value!

Docker, Telegraf, InfluxDB and Grafana

I have already presented various tutorials on docker monitoring. This time we will use Telegraf.

Project preparation

InfluxDB preparation

Telegraf preparation

… Read more about Telegraf on documentation page …

Note: The space after /etc is just because of security settings of my provider.

… after short time …

Grafana preparation

After login (admin:admin) you can add new Data Source.

Grafana InfluxDB Telegraf DataSource

Okay … all done … you can start to create Dashboards or search for existing Dashboards for import.

Docker, cAdvisor, InfluxDB and Grafana

In previous tutorials I showed the basics for docker monitoring with Prometheus and Jenkins. Here are now the basics with cAdvisor. Many steps are similar and can be implemented just as quickly.

Preparation

InfluxDB preparation

This time we start with InfluxDB, because cAdvisor needs it for connection!

cAdvisor preparation

Now we will use User, Password and Database name.Β You can find the documentation here.

After a while we can also see if cAdvisor sends metrics to InfluxDB.

prepare and run Grafana

After login (admin:admin) configure new DataSource for InfluxDB.

DataSource InfluxDB

When DataSource is configured we import the Grafana Dashboard. (ID: 1367)

Dashboard Search cAdvisor

… be patient (don’t press any button) …

Import cAdvisor Dashboard

Select already created DataSource (InfluxDB) and press β€œImport” button. Now you should see the Grafana Dashboard.

macOS, Docker, Prometheus and Grafana

I like Grafana … the dashboards are just cool! Here (again) a tutorial about docker monitoring. In less minutes you should be done.Β As a comment … for Linux and Windows you can do that too! There are only partial changes.

Prepare Project

Replace <yourLocalIP> with your IP. On Docker website you can find templates for Linux and Windows, too!

Configure Docker

This step is very easy. Just open Docker “Preferences” and specify in section “Daemon” -> “Advanced” the metrics-address. Just ensure that you use valid JSON!

macOS Docker Metrics

When you are done, press “Apply and Restart” button.

Prepare Prometheus

Note: The space after /etc is just because of security settings of my provider! Please remove the space.

prometheus WebUI

Just for fun you can create already some graphs in Prometheus.

prometheus Graph

Prepare and run Grafana

After login (admin:admin) configure new DataSource for Prometheus.

Grafana DataSource Prometheus

Import Dashboard (ID: 1229)

Grafana Import Dashboard

… enter ID 1229 …

Grafana Dashboard Search

… be patient (don’t press any button) …

Docker Engine Metrics Dashboard

Select already created DataSource (Prometheus) and press “Import” button. Now you should see the awesome Grafana Dashboard.