Docker registry and Let’s Encrypt

In a previous tutorial, I showed you how to setup a insecure Docker registry. Now we will use HTTPS via certificates from Let’s Encrypt and without some insecure registry settings.

Order dedicated host

If you have a host already, skip this section. If you looking for an good and cheap dedicated host, have a look on Dedibox.

Dedibox

After successful order you can start to install CentOS (Server distributions).

install os on Dedibox

When the OS installation is done, please take care for security! On tecmint.com you can find some cool guides “The Mega Guide To Harden and Secure CentOS 7“. On official Docker docs you will found all needed steps for your Docker CE installation.

Register and configure free domain

Let’s Encrypt need a domain! Register on Freenom and order new domain for free (.tk, .ml, .ga, .cf, .gq). If you have a domain already, skip this section.

free domain

Ensure your dns is configured correctly!

Freenom dns management

Create new Let’s Encrypt certificates

Login into your host via SSH and follow next steps. Attention, replace “demotesthost.tk” by your own domain!

Note: The space after /etc is just because of security settings by my provider!

Run your Docker registry

Now it’s time for push and pull a images

After download and open Finder, you should see similar files.

letsencrypt  CA certificates

Simply install both CA certificates via double-click.

letsencrypt certificate install

Optional you can check via “Keychain Access.app”.

Keychain Access.app

Now restart local docker and try again.

… next steps

So what about authentication? Currently everybody can upload/download images! What that means for security, should be clear. Please read the Docker docs about.

Setup learning environment for security testing

In a previous tutorial, I showed you how to set up a security learning environment quickly. Since there are some changes now (ex: Webswing), I’ll do the tutorial again. By the way i use macOS, some commands could be different but similar for Linux or Windows.

Requirements

  • Docker (17.12.0-ce)
  • Safari (11.02)
  • Firefox (58.0)

Search and download needed docker images

Run DVWA container

DVWA inside Browser

…Setup/create new database…

setup dvwa

Run ZAP container

ZAP inside browser

Please check via “Tools” -> “Options” -> “Local Proxies” the right configuration!

ZAP Proxy configuration

You need do use the non-routable meta address (0.0.0.0)!

Run ThreadFix container

Login into ThreadFix with “user/password

ThreadFix login

Create new team with application.

Configure Firefox

Configure manual proxy with your local IP and 2nd ZAP port (8090). Afterwards you enable checkbox “Use this proxy server for all protocols” and press OK button.

Firefox proxy configuration

When all the configurations have been made, you can temporarily quit Firefox.

Use environment

Start recording and save your XML report “Report -> Generate XML Report”.

Save ZAP XML report

Import XML report

Next steps

Now it’s time to study! If you need help, you can found a lot of training materials on Youtube.

Create a simple video test environment (Part 3)

Okay, now is time to see some command line tools to analysis videos. I selected 4 Open-Source applications (avprobe, mediainfo, mplayer, exiftool).

Specification

  • docker
  • git

Get ready for docker images

On Bitbucket I created a repository with needed Dockerfiles for fast usage. You can also choose the installation method.

mediainfo

Lets start with mediainfo. Here some information about on wikipedia.

mplayer

Second application is mplayer. Here the wikipedia link.

exiftool

Now we take a look on exiftool. Here the wikipedia article and the official documentation.

avprobe

Last but not least avprobe. Here the wikipedia article and detailed official documentation.

Compare tools by expecting specific result

I will not judge the applications against each other! But here a compare of complexity of commands and output for video duration.

Docker for Mac with insecure private registry

Sometimes you need an own Docker registry for testing purpose. Here a simple way to setup and use a private insecure registry. For production – don’t do that!

Requirements

Create insecure repository

SSH into your dedicated server…

Configure Docker for Mac

Start Docker for Mac and open “Preferences” – “Daemon”. Here just insert the IP plus specific port. When you are done press “Apply and Restart” button.

Docker for Mac Insecure registry

Build new image, tag and push

Create a new Dockerfile.

Now build the image, tag the image and push to your private registry.

Use insecure registry

Additional

You can see the images stored on your registry. Therefor SSH into your dedicated server again.

Shell linter evaluation and usage

Tomorrow, the 1st of August is a national holiday in Switzerland … So I do one day off and have some time. For a long time I wanted to deal with Shell lint. After some research, i found a few open-source tools. By the way … linters are being written for many programming languages and document formats.

Preparation

For evaluation i will not install the tools on my local system,… so Vagrant (with CentOS 7) is my choice.

Note: I created the Vagrant box “lupin/centos” via Packer … here my GitHub repository.

Shell option -n

Many shell’s already offer a very simple script analysis. The option -n read commands in script, but do not execute them (syntax check).

Okay … but not really what I want… (more details are welcome)

shlint and checkbashisms

I found the repository here.

Note: for both tools you should change the shebang to “#!/bin/sh”

For shlint… I don’t get it. For checkbashisms … good if will write portable Shell scripts.

bashate

I found it here on Pypi.

Nice … but not really all Standards.

Shellsheck

Shellcheck is known! Here the online service and here the repository.

I stay with that tool. Currently there are packages for almost every known OS.

Additional

Who knows me … knows that I do not like Installer and prefer Docker use. Here’s some fun.

😉 just for fun…

Docker, Telegraf, InfluxDB and Grafana

I have already presented various tutorials on docker monitoring. This time we will use Telegraf.

Project preparation

InfluxDB preparation

Telegraf preparation

… Read more about Telegraf on documentation page …

Note: The space after /etc is just because of security settings of my provider.

… after short time …

Grafana preparation

After login (admin:admin) you can add new Data Source.

Grafana InfluxDB Telegraf DataSource

Okay … all done … you can start to create Dashboards or search for existing Dashboards for import.

Docker, cAdvisor, InfluxDB and Grafana

In previous tutorials I showed the basics for docker monitoring with Prometheus and Jenkins. Here are now the basics with cAdvisor. Many steps are similar and can be implemented just as quickly.

Preparation

InfluxDB preparation

This time we start with InfluxDB, because cAdvisor needs it for connection!

cAdvisor preparation

Now we will use User, Password and Database name. You can find the documentation here.

After a while we can also see if cAdvisor sends metrics to InfluxDB.

prepare and run Grafana

After login (admin:admin) configure new DataSource for InfluxDB.

DataSource InfluxDB

When DataSource is configured we import the Grafana Dashboard. (ID: 1367)

Dashboard Search cAdvisor

… be patient (don’t press any button) …

Import cAdvisor Dashboard

Select already created DataSource (InfluxDB) and press “Import” button. Now you should see the Grafana Dashboard.

macOS, Docker, Prometheus and Grafana

I like Grafana … the dashboards are just cool! Here (again) a tutorial about docker monitoring. In less minutes you should be done. As a comment … for Linux and Windows you can do that too! There are only partial changes.

Prepare Project

Replace <yourLocalIP> with your IP. On Docker website you can find templates for Linux and Windows, too!

Configure Docker

This step is very easy. Just open Docker “Preferences” and specify in section “Daemon” -> “Advanced” the metrics-address. Just ensure that you use valid JSON!

macOS Docker Metrics

When you are done, press “Apply and Restart” button.

Prepare Prometheus

Note: The space after /etc is just because of security settings of my provider! Please remove the space.

prometheus WebUI

Just for fun you can create already some graphs in Prometheus.

prometheus Graph

Prepare and run Grafana

After login (admin:admin) configure new DataSource for Prometheus.

Grafana DataSource Prometheus

Import Dashboard (ID: 1229)

Grafana Import Dashboard

… enter ID 1229 …

Grafana Dashboard Search

… be patient (don’t press any button) …

Docker Engine Metrics Dashboard

Select already created DataSource (Prometheus) and press “Import” button. Now you should see the awesome Grafana Dashboard.