Simple Jenkins pipeline on AWS (Part 3) –

Okay,… The pipeline has already two steps “Build” and “Deploy” running, but the last step “Test” is missing. In this part I will show a simple example with Python, Selenium and Docker (standalone-chrome) for test step.

Preconditions

Install additional packages on AWS EC2

There is a need to install additional packages on AWS EC2 Linux instance (Jenkins).

# start ssh connection
$ ssh -i ~/.ssh/ExampleKeyPair.pem ec2-user@<EC2 IP|DNS>

# change to root user
$ sudo su -

# install python pip
$ easy_install pip

# install virtualenv
$ pip install virtualenv

# exit root and go back to ec2-user
$ exit

# exit ec2-user (ssh connection)
$ exit

Create new files and folder (Project/Repository)

You need to create a new directory called “test”. Inside that directory you will create a file “example.py” with following content.

#!/usr/bin/env python
import unittest
from selenium import webdriver


class ExampleTest(unittest.TestCase):

    def setUp(self):
        """Start web driver"""
        options = webdriver.ChromeOptions()
        options.add_argument('--no-sandbox')
        options.add_argument('--headless')
        options.add_argument('--disable-gpu')

        self.driver = webdriver.Remote('http://0.0.0.0:4444/wd/hub', options.to_capabilities())
        self.driver.get("APPLICATION_URL")

    def test_search_headline(self):
        """TestCase 1"""
        title = 'DemoPipeline'
        assert title in self.driver.title

    def test_search_text(self):
        """TestCase 2"""
        element = self.driver.find_element_by_tag_name('body')
        assert element.text == 'Hello world...'

    def tearDown(self):
        """Stop web driver"""
        self.driver.quit()

if __name__ == "__main__":
    unittest.main(verbosity=2)

When you are done you have to modify the “Jenkinsfile” and the bash script “test.sh”.

pipeline {
  agent any
  parameters {
    string(name: 'REPONAME', defaultValue: 'example/nginx', description: 'AWS ECR Repository Name')
    string(name: 'ECR', defaultValue: '237724776192.dkr.ecr.eu-central-1.amazonaws.com/example/nginx', description: 'AWS ECR Registry URI')
    string(name: 'REGION', defaultValue: 'eu-central-1', description: 'AWS Region code')
    string(name: 'CLUSTER', defaultValue: 'ExampleCluster', description: 'AWS ECS Cluster name')
    string(name: 'TASK', defaultValue: 'ExampleTask', description: 'AWS ECS Task name')
  }
  stages {
    stage('BuildStage') {
      steps {
        sh "./cicd/build.sh -b ${env.BUILD_ID} -n ${params.REPONAME} -e ${params.ECR} -r ${params.REGION}"
      }
    }
    stage('DeployStage') {
      steps {
        sh "./cicd/deploy.sh -b ${env.BUILD_ID} -e ${params.ECR} -c ${params.CLUSTER} -t ${params.TASK}"
      }
    }
    stage('TestStage') {
      steps {
        sh "./cicd/test.sh -c ${params.CLUSTER} -t ${params.TASK}"
      }
    }
  }
}

#!/usr/bin/env bash

## shell options
set -e
set -u
set -f

## magic variables
declare CLUSTER
declare TASK
declare TEST_URL
declare -r -i SUCCESS=0
declare -r -i NO_ARGS=85
declare -r -i BAD_ARGS=86
declare -r -i MISSING_ARGS=87

## script functions
function usage() {
  local FILE_NAME

  FILE_NAME=$(basename "$0")

  printf "Usage: %s [options...]\n" "$FILE_NAME"
  printf " -h\tprint help\n"
  printf " -c\tset esc cluster name uri\n"
  printf " -t\tset esc task name\n"
}

function no_args() {
  printf "Error: No arguments were passed\n"
  usage
  exit "$NO_ARGS"
}

function bad_args() {
  printf "Error: Wrong arguments supplied\n"
  usage
  exit "$BAD_ARGS"
}

function missing_args() {
  printf "Error: Missing argument for: %s\n" "$1"
  usage
  exit "$MISSING_ARGS"
}

function get_test_url() {
  local TASK_ARN
  local TASK_ID
  local STATUS
  local HOST_PORT
  local CONTAINER_ARN
  local CONTAINER_ID
  local INSTANCE_ID
  local PUBLIC_IP

  # list running task
  TASK_ARN="$(aws ecs list-tasks --cluster "$CLUSTER" --desired-status RUNNING --family "$TASK" | jq -r .taskArns[0])"
  TASK_ID="${TASK_ARN#*:task/}"

  # wait for specific container status
  STATUS="PENDING"
  while [ "$STATUS" != "RUNNING" ]; do
    STATUS="$(aws ecs describe-tasks --cluster "$CLUSTER" --task "$TASK_ID" | jq -r .tasks[0].containers[0].lastStatus)"
  done

  # get container id
  CONTAINER_ARN="$(aws ecs describe-tasks --cluster "$CLUSTER" --tasks "$TASK_ID" | jq -r .tasks[0].containerInstanceArn)"
  CONTAINER_ID="${CONTAINER_ARN#*:container-instance/}"

  # get host port
  HOST_PORT="$(aws ecs describe-tasks --cluster "$CLUSTER" --tasks "$TASK_ID" | jq -r .tasks[0].containers[0].networkBindings[0].hostPort)"

  # get instance id
  INSTANCE_ID="$(aws ecs describe-container-instances --cluster "$CLUSTER" --container-instances "$CONTAINER_ID" | jq -r .containerInstances[0].ec2InstanceId)"

  # get public IP
  PUBLIC_IP="$(aws ec2 describe-instances --instance-ids "$INSTANCE_ID" | jq -r .Reservations[0].Instances[0].PublicIpAddress)"

  TEST_URL="$(printf "http://%s:%d" "$PUBLIC_IP" "$HOST_PORT")"
}

function clean_up() {
  # stop container
  if [ "$(docker inspect -f {{.State.Running}} ChromeBrowser)" == "true" ]; then
    docker rm -f ChromeBrowser
  fi

  # delete virtualenv
  if [ -d .env ]; then
    rm -fr .env
  fi
}

function run_selenium_test() {
  local TEST_TEMPLATE
  local TEST_FILE

  # clean up
  clean_up

  # pull image (standalone-chrome)
  docker pull selenium/standalone-chrome

  # run docker container (standalone-chrome)
  docker run -d -p 4444:4444 --name ChromeBrowser selenium/standalone-chrome

  # create and activate virtualenv
  virtualenv .env && source .env/bin/activate

  # install Selenium
  pip install -U selenium

  # read test template into variable
  TEST_TEMPLATE=$(cat ./test/example.py)

  # replace string with URL
  TEST_FILE="${TEST_TEMPLATE/APPLICATION_URL/$TEST_URL}"

  # save into final test file
  echo "$TEST_FILE" > ./test/suite.py

  # execute test
  python -B ./test/suite.py

  # deactivate virtualenv
  deactivate
}

## check script arguments
while getopts "hc:t:" OPTION; do
  case "$OPTION" in
    h) usage
       exit "$SUCCESS";;
    c) CLUSTER="$OPTARG";;
    t) TASK="$OPTARG";;
    *) bad_args;;
  esac
done

if [ "$OPTIND" -eq 1 ]; then
  no_args
fi

if [ -z "$CLUSTER" ]; then
  missing_args '-c'
fi

if [ -z "$TASK" ]; then
  missing_args '-t'
fi

## run main function
function main() {
  get_test_url
  printf "Test Application URL: %s\n" "$TEST_URL"

  run_selenium_test
}

main

# exit
exit "$SUCCESS"

Ensure that “example.py” has all needed permission rights. $ chmod +x example.py Commit all changes now and wait that the Jenkins job gets triggered (or trigger manually).

jenkins trigger with parameters

That’s already all… your job should execute all steps. This part is done super fast. 😉

Some last words

There is a lot of space for improvements here, but I think you learned already much and had some fun. Some hints now:

you can add any other test methods by your self on this step (eq. Performance- and Security tests)
Unit tests and Static Code Analysis could executed on build step (before create image)
check out AWS ECS Services
use a proxy for Jenkins and enable SSL
create other pipelines and ECS clusters to enable staging
create “Lifecycle policy rules” on ECR
use Git Webhook’s to trigger the Jenkins jobs
add a post step in your Jenkins pipeline to store metrics and/or inform about build status

– Softwaretester –

knowledge is power

Simple Jenkins pipeline on AWS (Part 3)