Pimp my Kubernetes WebUI

There is a very easy way to pimp the Kubernetes WebUI with monitoring output. The whole thing we now realize super fast via Heapster, InfluxDB and Grafana.

Conditions

  • Installed and running Docker for Mac (edge)
  • Kubernetes enabled

Preparation

# list all pods
$ kubectl get pods --all-namespaces
...
NAMESPACE     NAME                                         READY     STATUS    RESTARTS   AGE
kube-system   kubernetes-dashboard-5bd6f767c7-f9w4j        1/1       Running   1          17d
...

# create port forward
$ kubectl port-forward kubernetes-dashboard-5bd6f767c7-f9w4j 8443:8443 --namespace=kube-system

# open WebUI in browser
$ open https://localhost:8443

# get token
$ kubectl -n kube-system get secret | grep deployment-controller-token
...
deployment-controller-token-s4xdg                kubernetes.io/service-account-token   3         17d
...

# show token
$ kubectl -n kube-system describe secret deployment-controller-token-s4xdg
...
token: XXXX
...

Now login to the WebUI with the token.

WebUI Token Login

Enable Monitoring

Download all 3 files from GitHub kubernetes/Heapster into your project. After download we need to modify a little bit and create deployment + service.

# edit heapster.yml
$ vim heapster.yml

# edit grafana.yml
$ vim grafana.yml

# edit influxdb.yml
$ vim influxdb.yml

Attention: The respective sections Services have to be adapted!
… But leave the rest of the content as is.

...
---
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    kubernetes.io/name: Heapster
  name: heapster
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 8082
  selector:
    k8s-app: heapster
...
---
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    kubernetes.io/name: monitoring-influxdb
  name: monitoring-influxdb
  namespace: kube-system
spec:
  ports:
  - port: 8086
    targetPort: 8086
  selector:
    k8s-app: influxdb
...
---
apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/name: monitoring-grafana
  name: monitoring-grafana
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 3000
  selector:
    k8s-app: grafana
# create resources from files
$ kubectl create -f heapster.yml
$ kubectl create -f influxdb.yml
$ kubectl create -f grafana.yml

That’s it already – our monitoring is enabled! Let’s take a look at everything.

# list all services (optional)
$ kubectl get services --all-namespaces

# show details of monitoring-grafana (NodePort)
$ kubectl describe services monitoring-grafana --namespace kube-system

# open Grafana in browser
$ open http://localhost:30703

Grafana Dashboards

Grafana Cluster Dashboard

Grafana Pod Dashboard

WebUI Dashboards

After a while it should look like this.

WebUI Workloads

WebUI Pods

Running Jenkins on Kubernetes (Docker for Mac)

Now we will deploy Jenkins-Docker on local Kubernetes. If you haven’t Kubernetes running yet, feel free to have a look on my previous tutorial. I will try to describe with very basic steps the tutorial. That’s may confusing for advanced peoples or experts but it should help beginner to get in that topic. For example, this tutorial uses 2 YAML files.

Preparation

# create new project
$ mkdir -p ~/Projects/KubernetesJenkins && cd ~/Projects/KubernetesJenkins

# create needed files
$ touch namespace.yml pod.yml

# modify namespace.yml
$ vim namespace.yml

# modify pod.yml
$ vim pod.yml
apiVersion: v1
kind: Namespace
metadata:
  name: qa-namespace
  labels:
    name: qa-namespace
apiVersion: v1
kind: Pod
metadata:
  name: jenkins.example.com
  labels:
    app: qa-jenkins-app
  namespace: qa-namespace
spec:
  containers:
    - name: jenkins
      image: jenkins/jenkins:lts-alpine
      ports:
        - containerPort: 8080

Let’s go – start Jenkins container on Kubernetes

# show nodes (optional)
$ kubectl get nodes

# create namespace
$ kubectl create -f ~/Projects/KubernetesJenkins/namespace.yml

# show namespaces (optional)
$ kubectl get namespaces --show-labels

# create pod
$ kubectl create -f pod.yml

# show pods of namespace
$ kubectl get pods --namespace qa-namespace

# show pod informations (optional)
$ kubectl describe pod jenkins.example.com --namespace qa-namespace

Open Jenkins in Browser

Jenkins is already running but you cannot access Jenkins without one important step! You need to configure the network routing. Probably the easiest option to do that is a simple port-forward.

# show ports in use (optional)
$ lsof -i -P | grep -i "listen"

# create port-forward to specific namespace
$ kubectl port-forward jenkins.example.com 8080:8080 --namespace=qa-namespace

# open browser (new terminal)
$ open http://localhost:8080

The 2nd way is to expose a service. This possibility is recommended only for local environments! For example on AWS you use load-balancer and there the way is a little bit different.

# expose pod as service
$ kubectl expose pod jenkins.example.com --namespace=qa-namespace --type=NodePort --name jenkins-service

# show services in namespace (optional)
$ kubectl get services --namespace qa-namespace

# show service informations (optional)
$ kubectl get service jenkins-service --namespace qa-namespace

# get node port
$ kubectl describe service jenkins-service --namespace qa-namespace | grep NodePort

# open browser (same terminal)
$ open http://localhost:32654

Whatever way you prefer, you need the initial admin password for Jenkins and/or you may need to see logs.

# show key for Jenkins activation
$ kubectl exec jenkins.example.com --namespace qa-namespace -- cat /var/jenkins_home/secrets/initialAdminPassword

# show logs of pod
$ kubectl logs -f jenkins.example.com --namespace qa-namespace

That’s it… Now you can use Jenkins.

CleanUp

If you want to clean up, proceed as follows.

# delete service
$ kubectl delete service jenkins-service --namespace qa-namespace

# list services (optional)
$ kubectl get services --namespace qa-namespace

# delete pod
$ kubectl delete pod jenkins.example.com --force --namespace qa-namespace

# list pods (optional)
$ kubectl get pods --namespace qa-namespace

# delete namespace
$ kubectl delete namespaces qa-namespace

# list namespaces (optional)
$ kubectl get namespaces

Kubernetes with Docker for Mac

The newer versions of Docker for Mac actually bring everything for the use of Kubernetes. Since the current documentation is not so optimal, I try it in my own way. Since I plan to further testing tutorials on this topic, this guide will serve as a basis.

Preparation

Kubernetes is currently only supported via Docker Edge. Caution, if you switch from stable to edge all Docker images and containers will be deleted! If you are already using the Edge version, skip the following steps 1 till 3.

Docker for Mac Version Stable

  1. Download Docker for Mac Edge Version… You can exit Docker for Mac while downloading.
  2. After successful download of DMG start the installation (Replace the old version).
  3. Start Docker and follow the instructions.
  4. Activate Kubernetes now via “Enable Kubernetes” checkbox and install the Kubernetes cluster. This can take a while, do not lose your patience!
  5. When the installation is finished you can check it.

Enable Kubernetes

Docker Version Edge with Kubernetes

Note, if you have already used minikube, you should now switch the cluster. You can switch between clusters at any time via GUI or command-line.

# show kubectl version
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Unable to connect to the server: dial tcp 192.168.99.100:8443: i/o timeout

# show current context
$ kubectl config current-context
minikube

# switch context
$ kubectl config use-context docker-for-desktop
Switched to context "docker-for-desktop".

# show kubectl version
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Now it’s a good time to know some more about current cluster, nodes, pods and namespaces. This will help to understand everything better!

# show cluster informations
$ kubectl cluster-info
Kubernetes master is running at https://localhost:6443
KubeDNS is running at https://localhost:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

# show nodes informations
$ kubectl get nodes
NAME                 STATUS    ROLES     AGE       VERSION
docker-for-desktop   Ready     master    14m       v1.9.6

# show pod informations
$ kubectl get pods
No resources found.

# show namespaces informations
$ kubectl get namespaces
NAME          STATUS    AGE
default       Active    23m
docker        Active    21m
kube-public   Active    23m
kube-system   Active    23m

As you can see, everything is working fine. The system is now ready for usage. By the way, have a look on your Docker images!

# list current Docker images (optional)
$ docker images
...

Deploying the Kubernetes Web UI Dashboard

Finally we deploy the Kubernetes Web UI Dashboard on our new Kubernetes Master as a Pod in namespace kube-system. The Dashboard is not installed/deployed by default. Although everything is possible via command-line, it can help to better understand and analyze the system.

# create a resource from a file
$ kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
secret "kubernetes-dashboard-certs" created
serviceaccount "kubernetes-dashboard" created
role "kubernetes-dashboard-minimal" created
rolebinding "kubernetes-dashboard-minimal" created
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created

# list pods in specific namespace
$ kubectl get pods --namespace=kube-system
NAME                                         READY     STATUS    RESTARTS   AGE
etcd-docker-for-desktop                      1/1       Running   0          46m
kube-apiserver-docker-for-desktop            1/1       Running   0          46m
kube-controller-manager-docker-for-desktop   1/1       Running   0          46m
kube-dns-6f4fd4bdf-f7pjw                     3/3       Running   0          47m
kube-proxy-c676q                             1/1       Running   0          47m
kube-scheduler-docker-for-desktop            1/1       Running   0          46m
kubernetes-dashboard-5bd6f767c7-f9w4j        1/1       Running   0          1m

# forward port to specific pod (Attention! Your pod may have a different name!)
$ kubectl port-forward kubernetes-dashboard-5bd6f767c7-f9w4j 8443:8443 --namespace=kube-system
Forwarding from 127.0.0.1:8443 -> 8443

# open Web UI Dashboard
$ open https://localhost:8443

Skip Authentication

You can skip authentication and jump directly to the dashboard. This step should may give you a hint. Never ever do the same in production!

Kubernetes Dashboard

That’s it already! Have a look on created dashboard and get familiar with your new Kubernetes environment.

PHP QA Tools and Docker Jenkins

This Tutorial is about some simple PHP QA Tools and Docker Jenkins. I will show near how to install PHP and PHP Composer in an Jenkins Alpine Linux Docker inclusive some needed Jenkins PlugIns.

Note

If you have an running Docker Container already which you cannot stop, you can install needed packages directly via:

# list containers (optional)
$ docker ps -a

# access running container as root
$ docker exec -u 0 -it <Container Name> sh

# install packages and exit container
...

Now you can use the same commented commands as provided via Dockerfile. Otherwise follow next steps.

Let’s go

# create new project
$ mkdir -p ~/Projects/DockerJenkins && cd ~/Projects/DockerJenkins/

# create Dockerfile and plugins.txt
$ touch Dockerfile plugins.xt

# modify Dockerfile
$ vim Dockerfile

# modify plugins.txt
$ vim plugins.txt
FROM jenkins/jenkins:lts-alpine

USER root

RUN apk update && apk upgrade

# install needed libary packages
RUN apk --no-cache add libssh2 libpng freetype libjpeg-turbo libgcc \
libxml2 libstdc++ icu-libs libltdl libmcrypt

# install needed PHP packages
RUN apk --no-cache add php7 php7-fpm php7-opcache php7-gd php7-pdo_mysql \
php7-mysqli php7-mysqlnd php7-mysqli php7-zlib php7-curl php7-phar \
php7-iconv php7-pear php7-xml php7-pdo php7-ctype php7-mbstring \
php7-soap php7-intl php7-bcmath php7-dom php7-xmlreader php7-openssl \
php7-tokenizer php7-simplexml php7-json

# Download and install composer installer
RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
RUN php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
RUN php composer-setup.php
RUN mv composer.phar /usr/local/bin/composer
RUN chmod +x /usr/local/bin/composer
RUN rm -f composer-setup.php

USER jenkins

# install plugins from plugins.txt
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
checkstyle:3.50
analysis-core:1.95
dry:2.50
pmd:3.50
violations:0.7.11

That was it! Now build the image, start and work with jenkins.

# build image from Dockerfile
$ docker build -t lupin/jenkins:lts-alpine .

# list images (optional)
$ docker images

# start container
$ docker run --name JenkinsPHP -p 8080:8080 lupin/jenkins:lts-alpine

Test

After starting, configuring and logging, you can see the already installed plugins in the Jenkins PlugIns!

Jenkins PlugIns

To test, you can create a simple freestyle job. Here you configure the repository, build steps and post-build actions. After a few runs, the results should be visible on the project side.

Jenkins Build Results

Docker registry and Let’s Encrypt

In a previous tutorial, I showed you how to setup a insecure Docker registry. Now we will use HTTPS via certificates from Let’s Encrypt and without some insecure registry settings.

Order dedicated host

If you have a host already, skip this section. If you looking for an good and cheap dedicated host, have a look on Dedibox.

Dedibox

After successful order you can start to install CentOS (Server distributions).

install os on Dedibox

When the OS installation is done, please take care for security! On tecmint.com you can find some cool guides “The Mega Guide To Harden and Secure CentOS 7“. On official Docker docs you will found all needed steps for your Docker CE installation.

Register and configure free domain

Let’s Encrypt need a domain! Register on Freenom and order new domain for free (.tk, .ml, .ga, .cf, .gq). If you have a domain already, skip this section.

free domain

Ensure your dns is configured correctly!

Freenom dns management

Create new Let’s Encrypt certificates

Login into your host via SSH and follow next steps. Attention, replace “demotesthost.tk” by your own domain!

# install epel-release
$ yum install -y epel-release

# install certbot
$ yum install -y certbot

# show default configuration
$ firewall-cmd --list-all

# open firewall ports 80, 443
$ firewall-cmd --permanent --add-port=80/tcp
$ firewall-cmd --permanent --add-port=443/tcp

# reload firewall
$ firewall-cmd --reload

# create needed certificates
$ certbot certonly --standalone --email admin@demotesthost.tk -d demotesthost.tk

# close firewall port 80
$ firewall-cmd --permanent --remove-port=80/tcp

# reload firewall
$ firewall-cmd --reload

# list certificates (optional)
$ ls -lahG /etc /letsencrypt/archive/demotesthost.tk/

# create new directory
$ mkdir -p /opt/certs

# copy needed certificates
$ cp /etc /letsencrypt/archive/demotesthost.tk/cert1.pem /opt/certs/
$ cp /etc /letsencrypt/archive/demotesthost.tk/privkey1.pem /opt/certs/

# list certificates (optional)
$ ls -la /opt/certs/
...
-rwxrwxrwx  1 root root 1797 Feb 27 17:17 cert1.pem
-rwxrwxrwx  1 root root 1704 Feb 27 17:17 privkey1.pem

Note: The space after /etc is just because of security settings by my provider!

Run your Docker registry

# search for image (optional)
$ docker search registry
NAME              DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
registry          The Docker Registry 2.0 implementation for s…   1853      [OK]                
...

# download and run container
$ docker run -d \
  --restart=always \
  --name registry \
  -v /opt/certs:/certs \
  -e REGISTRY_HTTP_SECRET=test1234 \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/cert1.pem \
  -e REGISTRY_HTTP_TLS_KEY=/certs/privkey1.pem \
  -p 443:5000 \
  registry:2

# check logs (optional)
$ docker logs -f registry

# check container (optional)
$ docker ps -a

# connection test from local to host (optional)
$ curl -v https://demotesthost.tk/v2/_catalog

Now it’s time for push and pull a images

# pull small alpine image
$ docker pull alpine

# tag alpine image
$ docker tag alpine demotesthost.tk/myalpine

# push image to registry (try)
$ docker push demotesthost.tk/myalpine
The push refers to repository [demotesthost.tk/myalpine]
Get https://demotesthost.tk/v2/: x509: certificate signed by unknown authority

# download needed certs
$ curl -O https://letsencrypt.org/certs/isrgrootx1.pem
$ curl -O https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem

# open directory in Finder
$ open .

After download and open Finder, you should see similar files.

letsencrypt  CA certificates

Simply install both CA certificates via double-click.

letsencrypt certificate install

Optional you can check via “Keychain Access.app”.

Keychain Access.app

Now restart local docker and try again.

# push image
$ docker push demotesthost.tk/alpine

# check repositories via curl (optional)
$ curl https://demotesthost.tk/v2/_catalog
{"repositories":["alpine"]}

# check image tags via curl (optional)
$ curl https://demotesthost.tk/v2/alpine/tags/list
{"name":"alpine","tags":["latest"]}

# delete local images
$ docker rmi alpine
$ docker rmi demotesthost.tk/alpine

# pull image from your registry
$ docker pull demotesthost.tk/alpine

… next steps

So what about authentication? Currently everybody can upload/download images! What that means for security, should be clear. Please read the Docker docs about.

Create phishing sites with SocialFish

With SocialFish you can easily and quickly create phishing sites. Of course, only for learning purposes! Here is a little tutorial to getting started.

Requirements

  • make, PHP, sudo
  • Python 2.7
  • Python PIP
  • Python virtualenv

Preparation

# create new project
$ mkdir -p ~/Projects/SocialFish && cd ~/Projects/SocialFish

# create Makefile
$ vim Makefile
VIRTUALENV_DIR = .env

.PHONY: destroy

CURRENT_DIR := $(shell pwd)
INTERPRETER = $(CURRENT_DIR)/$(VIRTUALENV_DIR)/bin
PATH := ${PATH}:$(INTERPRETER)/

help :
	@echo "Usage: $ make <target>"
	@echo " > create    : create project"
	@echo " > run       : run project"
	@echo " > destroy   : destroy project"

create :
	@echo "[RUN]: clone from git"
	@git clone https://github.com/UndeadSec/SocialFish.git
	@make env

destroy :
	@echo "[RUN]: destroy project"
	@rm -fr ./$(VIRTUALENV_DIR)/
	@rm -fr ./SocialFish/

env :
	@echo "[RUN]: create virtualenv"
	@virtualenv $(VIRTUALENV_DIR) && \
	. $(VIRTUALENV_DIR)/bin/activate
	@make deps

deps :
	@echo "[RUN]: install dependencies"
	@$(INTERPRETER)/pip install -r $(CURRENT_DIR)/SocialFish/requirements.txt

run :
	@echo "[RUN]: start SocialFish"
	@cd SocialFish && \
	../.env/bin/python SocialFish.py

Usage

Here an phishing example for GitHub.

# download and create
$ make create

# start SocialFish
$ make run

 [!] Do you agree to use this tool for educational purposes only? (y/n)
 SF > y

Select an option:

 [1] Facebook

 [2] Google

 [3] LinkedIn

 [4] Github

 [5] StackOverflow

 [6] WordPress

 [7] Twitter

 SF >  4


   _.-=-._     .-, 
 .'       "-.,' / 
(          _.  < 
 `=.____.="  `._\


 [*] Github module loaded. Building site...
Password:

 [*] Ngrok URL: https://5cf8cc89.ngrok.io

 [*] Waiting for credentials... 

PHP 7.1.7 Development Server started at Sun Feb 18 16:11:23 2018
Listening on http://127.0.0.1:80
Document root is /Users/lupin/Projects/SocialFish/SocialFish/Server/www
Press Ctrl-C to quit.

SocialFish

[Sun Feb 18 16:13:06 2018] 127.0.0.1:50603 [200]: /
[Sun Feb 18 16:13:07 2018] 127.0.0.1:50612 [404]: /opensearch.xml - No such file or directory
[Sun Feb 18 16:13:18 2018] 127.0.0.1:50618 [302]: /login.php
 [ CREDENTIALS FOUND ]:
 [EMAIL]: test@mail.de [PASS]: test1234

# destroy all
$ make destroy

😉 super easy …

Setup learning environment for security testing

In a previous tutorial, I showed you how to set up a security learning environment quickly. Since there are some changes now (ex: Webswing), I’ll do the tutorial again. By the way i use macOS, some commands could be different but similar for Linux or Windows.

Requirements

  • Docker (17.12.0-ce)
  • Safari (11.02)
  • Firefox (58.0)

Search and download needed docker images

# search DVWA image (optional)
$ docker search dvwa

# pull DVWA image
$ docker pull citizenstig/dvwa

# search ZAP image (optional)
$ docker search zap

# pull ZAP image
$ docker pull owasp/zap2docker-stable

# search ThreadFix image (optional)
$ docker search threadfix

# pull ThreadFix image
$ docker pull jmbmxer/threadfix

# list images (optional)
$ docker images
...
REPOSITORY                 TAG       IMAGE ID        CREATED         SIZE
owasp/zap2docker-stable    latest    40848e80b7fb    2 months ago    1.33GB
jmbmxer/threadfix          latest    b6f1907a61cd    22 months ago   941MB
citizenstig/dvwa           latest    c8312743bc09    3 years ago     478MB
...

Run DVWA container

# run DVWA container
$ docker run -d -p 8081:80 --name dvwa citizenstig/dvwa

# check DVWA logs for startup (optional)
$ docker logs -f dvwa

# get local ip
$ ipconfig getifaddr en0
...
192.168.192.39
...

# start DVWA in browser
$ open -a Safari http://192.168.192.39:8081/

DVWA inside Browser

…Setup/create new database…

setup dvwa

Run ZAP container

# create folder
$ mkdir -p /tmp/reports

# run ZAP container
$ docker run -u zap -i -p 8080:8080 -p 8090:8090 -v /tmp/reports:/home/zap/reports --name zap owasp/zap2docker-stable zap-webswing.sh

# start ZAP in Browser
$ open -a Safari http://localhost:8080/?anonym=true&app=ZAP

ZAP inside browser

Please check via “Tools” -> “Options” -> “Local Proxies” the right configuration!

ZAP Proxy configuration

You need do use the non-routable meta address (0.0.0.0)!

Run ThreadFix container

# run ThreadFix container
$ docker run -d -p 8443:8443 --name threadfix jmbmxer/threadfix start

# check ThreadFix logs for startup (optional)
$ docker logs -f threadfix
...
Jan 30, 2018 8:56:40 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 49986 ms
...

# start ThreadFix in Browser
$ open -a Safari https://localhost:8443/threadfix/

Login into ThreadFix with “user/password

ThreadFix login

Create new team with application.

Configure Firefox

# get local ip
$ ipconfig getifaddr en0
...
192.168.192.39
...

# open Firefox preferences
$ /Applications/Firefox.app/Contents/MacOS/firefox --preferences

Configure manual proxy with your local IP and 2nd ZAP port (8090). Afterwards you enable checkbox “Use this proxy server for all protocols” and press OK button.

Firefox proxy configuration

When all the configurations have been made, you can temporarily quit Firefox.

Use environment

# list all containers (optional)
$ docker ps -a

# open DVWA via Proxy
$ open -a firefox http://192.168.192.39:8081

Start recording and save your XML report “Report -> Generate XML Report”.

Save ZAP XML report

# open reports directory in finder
$ open /tmp/reports

Import XML report

Next steps

Now it’s time to study! If you need help, you can found a lot of training materials on Youtube.

Create a simple video test environment (Part 3)

Okay, now is time to see some command line tools to analysis videos. I selected 4 Open-Source applications (avprobe, mediainfo, mplayer, exiftool).

Specification

  • docker
  • git

Get ready for docker images

On Bitbucket I created a repository with needed Dockerfiles for fast usage. You can also choose the installation method.

# change directory (optional)
$ cd ~/Projects/

# clone repository
$ git clone https://bitbucket.org/Lupin3000/tinydockerapps ~/Projects/tinydockerapps

# change directory
$ cd ~/Projects/VideoTest/

# build docker image for mediainfo
$ docker build -t debian/mediainfo ~/Projects/tinydockerapps/mediainfo/

# build docker image for mplayer
$ docker build -t debian/mplayer ~/Projects/tinydockerapps/mplayer/

# build docker image for exiftool
$ docker build -t debian/exiftool ~/Projects/tinydockerapps/exiftool/

# build docker image for avprobe
$ docker build -t debian/avprobe ~/Projects/tinydockerapps/avprobe/

# check available images (optional)
$ docker images

mediainfo

Lets start with mediainfo. Here some information about on wikipedia.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --help

# run simple scan
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo demo.mp4

# run full scan
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo -f demo.mp4

# show aspect ratio
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Video;%DisplayAspectRatio%" demo.mp4

# show duration
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="General;%Duration/String3%" demo.mp4

# show audio format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Audio;%Format%" demo.mp4

# show resolution and codec
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Inform="Video;Resolution=%Width%x%Height%\nCodec=%CodecID%" demo.mp4

# list all possible file parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --info-parameters | less

# create XML report (all internal tags)
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo -f --Output=XML demo.mp4

# show mediatrace info
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo --Details=1 demo.mp4

# create report file
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mediainfo demo.mp4 --LogFile="Report.log"

mplayer

Second application is mplayer. Here the wikipedia link.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer --help

# show all properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4

# show all video properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep VIDEO

# show all audio properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep AUDIO

# show video format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/mplayer -vo null -ao null -frames 0 -identify demo.mp4 | grep ID_VIDEO_FORMAT

exiftool

Now we take a look on exiftool. Here the wikipedia article and the official documentation.

# show all parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool demo.mp4

# show all parameters sort by group (including duplicate and unknown tags)
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -a -u -g1 demo.mp4

# show friendly parameters
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -s -G demo.mp4

# show Height and Width
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*source*image*' demo.mp4

# show audio format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*Audio*Format*' demo.mp4

# show video duration
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool '-*Duration*' demo.mp4 | head -1

# create json output with specific values
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -j -VideoFrameRate -MediaDuration demo.mp4 > report.json

# create csv report file with specific values
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/exiftool -csv -FileSize -ImageWidth -ImageHeight -AudioFormat -AudioChannels demo.mp4 > report.csv

avprobe

Last but not least avprobe. Here the wikipedia article and detailed official documentation.

# list help
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe --help

# list available formats
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -formats

# list available codecs
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -codecs

# show all properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe demo.mp4

# show stream properties in json format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -of json -loglevel quiet -show_streams demo.mp4

# show specific properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -show_format -show_streams -pretty demo.mp4

# show size properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -show_entries format=size demo.mp4

# show duration and size properties
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -loglevel quiet -show_entries format=duration,size demo.mp4

# show duration and size properties in json format
$ docker run --rm -i -v ~/Projects/VideoTest/:/mnt debian/avprobe -of json -loglevel quiet -show_entries format=duration,size demo.mp4

Compare tools by expecting specific result

I will not judge the applications against each other! But here a compare of complexity of commands and output for video duration.

# get duration by exiftool
$ exiftool -s -s -s  -MediaDuration demo.mp4
...
0:01:04

# get duration by mediainfo
$ mediainfo --Inform="General;%Duration/String3%" demo.mp4
...
00:01:04.884

# get duration by avprobe
$ avprobe -v error -sexagesimal -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 demo.mp4
...
0:01:04.884000

# get duration by mplayer
$ mplayer -vo null -ao null -frames 0 -nolirc -identify demo.mp4 | grep ID_LENGTH | cut -d'=' -f2
...
64.88