Jenkins and Sitespeed.io

While surfing the internet I stumbled across Sitespeed.io. It’s a amazing collection of Open Source Tools, which make performance measuring for developers and testers super easy. I tried it out and was immediately impressed. Here’s a little tutorial on how to use Jenkins and Sitespeed.

Requirements

  • Docker (latest)

Environment setup

With minimal 2 commands the environment (via Docker) is already created. Most of the time will be needed for the plugins installation.

On setup wizard finish: unlock Jenkins, install the suggested plugins, create an account and finish the instance configuration.

Jenkins permissions to /var/run/docker.sock

Before you start with Jenkins job configuration, ensure that user jenkins has permissions to /var/run/docker.sock.

Jenkins job configuration

When Jenkins is ready (restarted), install the HTML Publisher PlugIn (no restart after installation of plugin required).

Jenkins HTML Publisher Plugin

Create a new free-style project named SiteSpeed.

Jenkins SiteSpeed Project

Attention: You need to specify later the absolute path to the local directory /target/workspace/SiteSpeed. If you do not know how, press save and start the build without any job information (empty job configuration) and follow the optional instructions.

In my case the path is: “/Users/steffen/Projects/Sitespeed/target/workspace/SiteSpeed”. Under job configuration section “Build” enable “Execute shell” and paste following command.

Via Post-Build-Action: Publish HTML reports you can enter the report very simple from the job project page.

Jenkins SiteSpeed Job Configuration

Save everything and run the job. After a short time you can look at the HTML report. See “Pages” > “https://www.sitespeed.io/” for screenshots, HAR and video files. On the website of sitespeed.io is a very detailed documentation and many more examples. Have fun!

Create QA dashboards with Grafana (Part 1)

Since I have my new role (Head of QA), many employees constantly want metrics from me. That means a lot of work for me. But since I do not always want to deal with such things, I have searched for a simpler way. So the question was – how can I deliver this data at any time and possibly from different sources (eq. JIRA, pipelines, test results, Salesforce, etc.)? Hmmm … Grafana is awesome – not only for DevOps! So in this tutorial series, I’d like to show you how to create nice and meaningful dashboards for your QA metrics in Grafana.

What you need?

  • Docker installed (latest version)
  • Bash (min. 3.2.x)

Prepare the project

In order to create dashboards in Grafana, you need a small environment (Grafana/InfluxDB) as well as some data. The next steps will help you to create them. The environment/services are simulated by docker containers. For the fictitious data, just use the bash script which I created for this tutorial.

Now copy/paste the content of the two files with your favorite editor. The content of docker-compose.yml.

And here the content of CreateData.sh.

Start environment and create data

Once the project and the files have been created, you can build and start the environment. For this you use Docker Compose.

In next step you create the InfluxDB databases (incl. fictitious measurements, series and data) via Bash script.

Note: You could use influx command to administrate InfluxDB directly.

Okay the environment preparation is done. Now start Grafana in browser.

New Grafana Login

The default username and password is “admin:admin“. Note, if you use docker-compose down you have to repeat most of steps like data creation. Better use docker-compose stop! … See you in 2nd part – where we add data sources and create dashboards.

File encryption/decryption using GPG

There are just too many people and organizations who are interested in our data. Thus, the secure transmission of data is important. Through encryption/decryption, data can be protected from access by third parties. There are already very long easy ways for the encryption/decryption but I have to find again and again that these are quite unknown. Herewith a little tutorial where I want to show possibilities by means of GPG.

Requirements

  • Docker (latest)

Environment preparation

By means of two Docker containers, we now want to simulate 2 persons who exchange the encrypted data.

Container (user_a)

Container (user_b)

No passphrase prompt

If you want to use the encryption/decryption without prompt, for example in a bash script, you can use the following options. Depending on the version, it can come to a distinction. Option 1 is by default not available in the Docker containers.

Multiple files

You can also use a simple loop to encrypt/decrypt multiple files. Please note the available GPG version/options. Here now a simple example without prompt.

Encryption and Decryption via keys

Container (user_a)

Container (user_b)

Both public keys are available.

Both clients need to import the public key from other.

Our user_a now encrypt data.

User_b now decrypt data.

I hope that you have found an entry point into the topic and I have woken up your interest.

Unseal Vault with PGP

In this tutorial I will show an example for unsealing Vault using GPG. We generate for two users the keys and each user will use them to unseal. For the storage we use Consul.

Conditions

Host Preparation

First we need to setup, configure and start Consul and Vault.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

Do not stop and/or close any terminal sessions!

Your project folder now should look like this:

Client Preparation

As I wrote – we need to simulate two users. Now to the Docker client’s…

Both client’s need similar configuration, so please execute the following steps on both containers.

Your project folder now should look like this:

Initialize and Unseal Vault

On the host we initialize the Vault and share unseal key’s back to the client’s.

Note: Save now all keys and share the correspondending <unseal keys> to the client’s!

Now our client’s can start the unseal of Vault. Even here, please execute the following steps on both containers.

Just for information

We configured both services (Consul and Vault) with WebUI.

Use the “Initial Root Token” to login into Vault’s WebUI.

Hashicorp Vault SSH OTP

With Vault’s SSH secret engine you can provide an secure authentication and authorization for SSH. With the One-Time SSH Password (OTP) you don’t need to manage keys anymore. The client requests the credentials from the Vault service and (if authorized) can connect to target service(s). Vault will take care that the OTP can be used only once and the access is logged. This tutorial will provide needed steps on a simple Docker infrastructure. Attention, in that tutorial Vault and Vault-SSH-Helper are running in Development Mode – don’t do that in production!

Conditions

  • Docker installed (latest CE)

Vault server

Let’s start and prepare the vault service.

Don’t stop or close terminal session! Open new terminal. Note: The IP’s I use in this tutorial may be different to yours.

Target server

Now we create and configure the target service.

Note: Because of the security settings of my provider, spaces are after “etc”. Please delete it after copy/paste.

Pam SSHD configuration (on target server)

SSHD configuration (on target server)

Client server

Last container is for simulating a client.

Usage

Most work is already done. Now we use the demo environment.

Note: Because of the security settings of my provider, spaces are after “root”. Please delete it after copy/paste.

Running Jenkins on Kubernetes (Docker for Mac)

Now we will deploy Jenkins-Docker on local Kubernetes. If you haven’t Kubernetes running yet, feel free to have a look on my previous tutorial. I will try to describe with very basic steps the tutorial. That’s may confusing for advanced peoples or experts but it should help beginner to get in that topic. For example, this tutorial uses 2 YAML files.

Preparation

Let’s go – start Jenkins container on Kubernetes

Open Jenkins in Browser

Jenkins is already running but you cannot access Jenkins without one important step! You need to configure the network routing. Probably the easiest option to do that is a simple port-forward.

The 2nd way is to expose a service. This possibility is recommended only for local environments! For example on AWS you use load-balancer and there the way is a little bit different.

Whatever way you prefer, you need the initial admin password for Jenkins and/or you may need to see logs.

That’s it… Now you can use Jenkins.

CleanUp

If you want to clean up, proceed as follows.

Kubernetes with Docker for Mac

The newer versions of Docker for Mac actually bring everything for the use of Kubernetes. Since the current documentation is not so optimal, I try it in my own way. Since I plan to further testing tutorials on this topic, this guide will serve as a basis.

Preparation

Kubernetes is currently only supported via Docker Edge. Caution, if you switch from stable to edge all Docker images and containers will be deleted! If you are already using the Edge version, skip the following steps 1 till 3.

Docker for Mac Version Stable

  1. Download Docker for Mac Edge Version… You can exit Docker for Mac while downloading.
  2. After successful download of DMG start the installation (Replace the old version).
  3. Start Docker and follow the instructions.
  4. Activate Kubernetes now via “Enable Kubernetes” checkbox and install the Kubernetes cluster. This can take a while, do not lose your patience!
  5. When the installation is finished you can check it.

Enable Kubernetes

Docker Version Edge with Kubernetes

Note, if you have already used minikube, you should now switch the cluster. You can switch between clusters at any time via GUI or command-line.

Now it’s a good time to know some more about current cluster, nodes, pods and namespaces. This will help to understand everything better!

As you can see, everything is working fine. The system is now ready for usage. By the way, have a look on your Docker images!

Deploying the Kubernetes Web UI Dashboard

Finally we deploy the Kubernetes Web UI Dashboard on our new Kubernetes Master as a Pod in namespace kube-system. The Dashboard is not installed/deployed by default. Although everything is possible via command-line, it can help to better understand and analyze the system.

Skip Authentication

You can skip authentication and jump directly to the dashboard. This step should may give you a hint. Never ever do the same in production!

Kubernetes Dashboard

That’s it already! Have a look on created dashboard and get familiar with your new Kubernetes environment.

PHP QA Tools and Docker Jenkins

This Tutorial is about some simple PHP QA Tools and Docker Jenkins. I will show near how to install PHP and PHP Composer in an Jenkins Alpine Linux Docker inclusive some needed Jenkins PlugIns.

Note

If you have an running Docker Container already which you cannot stop, you can install needed packages directly via:

Now you can use the same commented commands as provided via Dockerfile. Otherwise follow next steps.

Let’s go

That was it! Now build the image, start and work with jenkins.

Test

After starting, configuring and logging, you can see the already installed plugins in the Jenkins PlugIns!

Jenkins PlugIns

To test, you can create a simple freestyle job. Here you configure the repository, build steps and post-build actions. After a few runs, the results should be visible on the project side.

Jenkins Build Results