Virtualization

RPLidar A1 with ROS Melodic on Ubuntu 18.04

If you are (for example) an owner of NVIDIA Jetson Nano Developer Kit and RPLidar, you can use ROS Melodic to realize obstacle avoidance or simultaneous localization and mapping (SLAM). Here is a beginner’s tutorial for installing the necessary software.

Requirements

Objective

Installing ROS Melodic on Ubuntu 18.04 with Catkin Workspace and use of Python 2.7 for RPLIDAR A1.

Preparation

If you already have the necessary packages installed, you can skip this step.

# update & upgrade
$ sudo apt update && sudo apt upgrade -y

# install needed packages
$ sudo apt install -y git build-essential
Install package

If you run Ubuntu in VirtualBox it is recommended to install the Extension Pack and the VirtualBox Guest Additions.

# install needed packages
$ sudo apt install -y dkms linux-headers-$(uname -r)

# Menu -> Devices -> Insert Guest Additions Image

# reboot system
$ sudo reboot

Install ROS Melodic

For Ubuntu 18.04 you need the latest version ROS 1 (Robot Operating System) Melodic. The packages are not included in the standard sources.

Hint: If you still want to use a newer version of Ubuntu (like 20.04) you need ROS 2 Noetic!

Note: By default the ROS Melodic is using Python 2.7. For higher Python version the following description is not working! But it would also be possible to use Python 3.x, only the steps for installation are little different.

Add ROS repository

The ROS Melodic installation is relatively easy but can take a while depending on the internet connection, as there are many packages to be installed.

# update and install packages
$ sudo apt update && sudo apt install -y ros-melodic-desktop-full python-rosdep python-rosinstall python-rosinstall-generator python-wstool

# verify installation (optional)
$ ls -la /opt/ros/melodic/
add bashrc source
# initialize & update
$ sudo rosdep init && rosdep update

# show ros environment variables (optional)
$ printenv | grep ROS

Create a Catkin workspace

You now need the Catkin Workspace with the necessary RPLIDAR packages/binaries (RPLIDAR SDK from Slamtec).

# create and change into directory
$ mkdir -p $HOME/catkin_ws/src && cd $HOME/catkin_ws/src

# clone Git repository
$ git clone https://github.com/Slamtec/rplidar_ros.git

# change directory
$ cd $HOME/catkin_ws/

# build workspace
$ catkin_make

# verify directory content (optional)
$ ls -la devel/

# refresh environment variables
$ source devel/setup.bash

# verify path variable (optional)
$ echo $ROS_PACKAGE_PATH

# build node
$ catkin_make rplidarNode

Okay, that’s it… Wasn’t very difficult, but a lot of steps.

Start ROS Melodic

Now connect the RPLIDAR device. If you use VirtualBox, pass through the USB from host to guest.

# list USB device and verify permissions
$ ls -la /dev | grep ttyUSB

# change permissions
$ sudo chmod 0666 /dev/ttyUSB0

From now on you need two terminals! Once you have successfully gotartet roscore in the first terminal, switch to the second terminal.

# change directory
$ cd $HOME/catkin_ws/

# launch roscore
$ roscore

Note: Don’t close the first terminal though or don’t stop the running process!

# change directory
$ cd $HOME/catkin_ws

# refresh environment variables
$ source $HOME/catkin_ws/devel/setup.bash

# run UI
$ roslaunch rplidar_ros view_rplidar.launch

Sorry for using screenshots but my provider doesn’t allow the use of certain words or characters. 🙁

VirtualBox USB passthrough

Yes, this topic is very easy via VirtualBox UI and many people would not have in mind to do these actions with command line. But as I do mostly all tasks while development via terminal, I thought some other persons are interested in the solution, too. So here now a short tutorial to attach/detach USB devices on VirtualBox machines.

Requirements

Objective

Learn and understand the basics for command line USB passthrough for VirtualBox.

Gather informations of USB devices

In a first step it’s important to gather all informations about possible USB devices. There are 3 important values ​​to be determined.

  • UUID
  • VendorId
  • ProductId

In order to better understand which devices are involved, the product name is also an advantage.

# list all usb devices
$ VBoxManage list usbhost

# list all usb devices with specific information
$ VBoxManage list usbhost | grep -i "UUID\|VendorId\|ProductId\|Product\|Current State"

The value of “Current State” can provide you the information if a USB device is in usage or captured by a VM already.

Analysis of VM’s

In the second step, it is advisable to check whether the respective USB standard is configured in the target VM (and whether USB is turned on). There are currently 3 options that VirtualBox supports.

  • USB 1.1: OHCI
  • USB 2.0: OHCI + EHCI
  • USB 3.0: xHCI

Note: My VM is named Debian in all examples, please change this value for your specific needs!

# show all informations for specific vm
$ VBoxManage showvminfo "Debian"

# show usb status for specific vm
$ VBoxManage showvminfo "Debian" | grep -i "ohci\|ehci\|xhci"

Add/remove USB devices

In many cases, USB 1.1 is enabled by default and deactivation leads to a misconfiguration. But there are, as almost always, exceptions.

  • USB 1.1: --usb on|off
  • USB 2.0: --usbehci on|off
  • USB 3.0: --usbxhci on|off

Note: For these kind of configurations the VM should be stopped (turned off).

# enable USB 2.0 for specific vm
$ VBoxManage modifyvm "Debian" --usbehci on

# disable USB 2.0 for specific vm
$ VBoxManage modifyvm "Debian" --usbehci off

# enable USB 3.0 for specific vm
$ VBoxManage modifyvm "Debian" --usbxhci on

# disable USB 3.0 for specific vm
$ VBoxManage modifyvm "Debian" --usbxhci off

If you know the UUID (see first step), you can now attach/detach the respective USB devices.

Note: You can make the following changes during the VM is running. Don’t forget to change the UUID value from my examples.

# attach specifc usb device to specifc vm
$ VBoxManage controlvm "Debian" usbattach "076866d4-436e-424d-be64-660c23248f37"

# detach specifc usb device to specifc vm
$ VBoxManage controlvm "Debian" usbdetach "076866d4-436e-424d-be64-660c23248f37"

Nothing more is needed for this very simple task. But you can save some time and automate these steps with the following section!

Usage of USB filters

VirtualBox USB filters will help you to automatically assign USB devices to your VM. With other words – As soon as the VM starts (or is started) and the USB device is recognized by VirtualBox (and it’s not in use by the host), it will be made available to the guest.

Additionally it’s important to understand that there are two kind of filters (global and machine specifc). Global filters (IHostUSBDeviceFilter) are applied before any machine specific filter (IUSBDeviceFilter).

  • --target <uuid|vmname>|global
  • --action ignore|hold (global filters only)
  • --active yes|no

In this tutorial I focus only on machine specific filters.

# verify for filter on specific vm
$ VBoxManage showvminfo "Debian" | awk '/^USB Device Filters:/,/^Available remote USB devices:/'

Important is the value “index” from 0 to n! If you create the first filter – the index starts with an 0, the second filter with an 1 and so on. To add a filter you must provide the VendorId and ProductId (as minimum) from first section (values inside the brackets), the name is can be chosen free.

Note: you have to adjust the following values in my examples ​​for your environment.

# show help for usb filters
$ VBoxManage usbfilter --help

# create usb filter for specifc vm
$ VBoxManage usbfilter add 0 --target "Debian" --name 'YARD Stick One' --vendorid 1d50 --productid 605b --remote no

# modify usb filter for specifc vm
$ VBoxManage usbfilter modify 0 --target "Debian" --name 'YARD Stick One' --vendorid 1d50 --productid 605b --remote no --manufacturer 'great scott gadgets'

# delete usb filter for specifc vm
$ VBoxManage usbfilter remove 0 --target "Debian"

USB filters are a very nice feature of VirtualBox, just play around and do some further research on it – I’m sure you will love it!

PyCharm CE remote execution

Since I’m back to develop more in Python and need to execute my Python scripts also on remote machines – I asked my self: “How can I do with PyCharm CE and without any Plugin?” The solution was very easy and this tutorial is about.

Requirements

  • VirtualBox VM (e.g. Linux Debian incl. user and ssh server configured)
  • PyCharm CE installed

Note: Of course you could also use any other option then a Debian vm. Just adapt the tutorial steps for your specific needs.

Objective

Use terminal and external tools of PyCharm CE to execute Python scripts on remote machines.

Introduction

For this tutorial I’m using a VirtualBox VM (Debian Linux) which do have only a NAT interface configured. So I need to enable Port-forwarding for SSH. If this is not the case for you – you can jump over to the section where I explain the important steps in PyCharm.

Analysis of stopped VM’s

As a first step I check my VM settings (just to verify).

# list all vm's (optional)
$ VBoxManage list vms

# list all vm's and grep for name & nic rules
$ VBoxManage list -l vms | grep -i "Name:\|NIC 1 Rule"

Analysis of running VM’s

I don’t know the IP – so I do start the VM and check their all settings.

# start specific vm
$ VBoxManage startvm "Debian" --type headless

# list all running vm's (optional)
$ VBoxManage list runningvms

# list all running vm's and grep for name & nic rules (optional)
$ VBoxManage list runningvms -l | grep -i "Name:\|NIC Rule"

# list specific vm Nic informations (optional)
$ VBoxManage showvminfo "Debian" | grep -i "NIC"

# get IPv4 information of specific vm
$ VBoxManage guestproperty get "Debian" "/VirtualBox/GuestInfo/Net/0/V4/IP"

Add Port-forwarding

Now I know the IP (which in my case will not change) and can enable the Port-forwarding for SSH easily.

# shutdown specific vm
$ VBoxManage controlvm "Debian" poweroff

# add port-forwarding rule to specific vm
$ VBoxManage modifyvm "Debian" --natpf1 "SSH-PW,tcp,127.0.0.1,2222,10.0.2.15,22"

# list specific vm Nic informations (optional)
$ VBoxManage showvminfo "Debian" | grep -i "NIC 1 Rule"

# start specific vm
$ VBoxManage startvm "Debian" --type headless

# test ssh connection (optional)
$ ssh -p 2222 lupin@127.0.0.1 -v -C 'whoami && exit'

Note: On my VM (Debian) the user is named “lupin” this will be different for you! Also Openssh-server is enabled and I added my public ssh key on VM (authorized_keys).

PyCharm remote execution

As I also like and use the feature “scratches” of PyCharm, I will show first the remote execution of these files.

Prepare a scratch

I prepare a simple Python script scratch, which just prints out environment variables (same code I use later inside project example).

import os

print('hello world', os.environ)

To find the absolute path for my scratch.py – I run it. On terminal the path will be visible.

PyCharm run scratches

After I know the path I run some simple commands in PyCharm Terminal.

# execute local script via SSH on remote system
$ ssh -p 2222 user@127.0.0.1 -C "/usr/bin/python" < "path/to/scratches/scratch.py"

# same as above but with unbuffered binary stdout and stderr
$ ssh -p 2222 user@localhost -C "/usr/bin/python" -u - < "./test_remote.py"

It works perfectly fine.

Note: Please replace the value “user” in all ssh examples!

PyCharm CE remote scratch execution

Project files

For all other scripts in the PyCharm project I don’t want to type the terminal commands always! So I’m using the feature “External Tools”. To do so, I add a new item and use the built-in variables (Macros) of PyCharm.

PyCharm CE add external tool

You simply give a Name Run on Remote, optional a short description, for Program your shell /bin/zsh and on Arguments --login -c "ssh user@localhost -p 2222 -C /usr/bin/python -u - < "$FilePah$. If you press button “OK” the input will be saved. The value for Working Directory $ProjectFileDir$ will be set (normally) automatically.

Now you can use the context menu to execute your script over SSH on remote machine.

PyCharm CE external tool remote execution

Get IP of headless Virtualbox VM

This short article will describe how you will get quickly the IP for an headless running Virtualbox VM. For demonstration purpose, I have assigned an “Bridged Interface” on NIC 1. So the IP is dynamically allocated (IP address is being assigned by DHCP).

# start VM headless (if not running)
$ VBoxManage startvm --type headless "vm name"

# check VM state (optional)
$ VBoxManage showvminfo "vm name" | grep "State"

# show IP
$ VBoxManage guestproperty get "vm name" "/VirtualBox/GuestInfo/Net/0/V4/IP"
...
Value: 172.20.10.6
...

That was super easy … now you can connect via SSH (if SSH service is running).

# start SSH connection
$ ssh -C4 remote_user@172.20.10.6

Vagrant and Vault

I was a little surprised why there is no Vagrant plug-in for Vault. Then I thought no matter, because the Vagrantfile is actually a Ruby script. Let me try it. I have to say right away that I’m not a Ruby developer! But here is my solution which has brought me to the goal.

Prerequisite

  • latest Vault installed (0.11.0)
  • latest Vagrant installed (2.1.3)

Prepare project and start Vault

# create new project
$ mkdir -p ~/Projects/vagrant-vault && cd ~/Projects/vagrant-vault

# create 2 empty files
$ touch vagrant.hcl Vagrantfile

# start Vault in development mode
$ vault server -dev

Here my simple vagrant policy (don’t do that in production).

path "secret/*" {
  capabilities = ["read", "list"]
}

And here is my crazy and fancy Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'net/http'
require 'uri'
require 'json'
require 'ostruct'

################ YOUR SETTINGS ####################
ROLE_ID = '99252343-090b-7fb0-aa26-f8db3f5d4f4d'
SECRET_ID = 'b212fb14-b7a4-34d3-2ce0-76fe85369434'
URL = 'http://127.0.0.1:8200/v1/'
SECRET_PATH = 'secret/data/vagrant/test'
###################################################

def getToken(url, role_id, secret_id)
  uri = URI.parse(url + 'auth/approle/login')
  request = Net::HTTP::Post.new(uri)
  request.body = JSON.dump({
    "role_id" => role_id,
    "secret_id" => secret_id
  })

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    token = result.auth.client_token
    return token
  else
    return ''
  end
end

def getSecret(url, secret_url, token)
  uri = URI.parse(url + secret_url)
  request = Net::HTTP::Get.new(uri)
  request["X-Vault-Token"] = token

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  if response.code == "200"
    result = JSON.parse(response.body, object_class: OpenStruct)
    return result
  else
    return ''
  end
end

token = getToken(URL, ROLE_ID, SECRET_ID)

unless token.to_s.strip.empty?
  result = getSecret(URL, SECRET_PATH, token)
  unless result.to_s.strip.empty?
    sec_a = result.data.data.secret_a
    sec_b = result.data.data.secret_b
  end
else
  puts 'Error - please check your settings'
  exit(1)
end

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.post_up_message = 'Secret A:' + sec_a + ' - Secret B:' + sec_b
end

Configure Vault

# set environment variables (new terminal)
$ export VAULT_ADDR='http://127.0.0.1:8200'

# check status (optional)
$ vault status

# create simple kv secret
$ vault kv put secret/vagrant/test secret_a=foo secret_b=bar

# show created secret (optional)
$ vault kv get --format yaml -field=data secret/vagrant/test

# create/import vagrant policy
$ vault policy write vagrant vagrant.hcl

# show created policy (optional)
$ vault policy read vagrant

# enable AppRole auth method
$ vault auth enable approle

# create new role
$ vault write auth/approle/role/vagrant token_num_uses=1 token_ttl=10m token_max_ttl=20m policies=vagrant

# show created role (optional)
$ vault read auth/approle/role/vagrant

# show role_id
$ vault read auth/approle/role/vagrant/role-id
...
99252343-090b-7fb0-aa26-f8db3f5d4f4d
...

# create and show secret_id
$ vault write -f auth/approle/role/vagrant/secret-id
...
b212fb14-b7a4-34d3-2ce0-76fe85369434
...

Run it

# starts and provisions the vagrant environment
$ vagrant up

😉 … it just works

Quick and dirty sync folders for Vagrant

Background

In our company we have different development teams working with same Vagrant boxes. As a challenge they need different sync folder locations – even inside teams. I’m just too lazy to provide and maintain all Vagrantfile templates for their needs. So i provide a quick and dirty solution for them.

Example

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  # box name
  config.vm.box = "lupin/centos7"

  # Disable replacing insecure key
  config.ssh.insert_key = false

  # Disable default synced folder
  config.vm.synced_folder ".", "/vagrant", disabled: true

  # Synced folder(s) by argument
  if !ENV['mount'].nil?

    loop do
      puts "local folder: "
      local_dir = STDIN.gets.chomp
      puts "target folder: "
      target_dir = STDIN.gets.chomp

      if local_dir.empty? || target_dir.empty?
        break
      end

      config.vm.synced_folder local_dir, target_dir
    end

  end

 # VirtualBox specific configuration
  config.vm.provider "virtualbox" do |vb|
    vb.gui = false
    vb.memory = "1024"
    vb.cpus = 1
    vb.name = "example_vm"
  end

end

Usage

# simple command
$ vagrant up

# check for synced folder(s)
$ vagrant ssh -c 'ls -la /'

# destroy environment
$ vagrant halt

# run with argument (mount)
$ mount=yes vagrant up
...
local folder:
src_a <ENTER>
target folder:
/vagrant_a <ENTER>
local folder:
src_b <ENTER>
target folder:
/vagrant_b <ENTER>
local folder:
<ENTER>
target folder:
<ENTER>
...

# show content
$ vagrant ssh -c 'cat /vagrant_a/SomeTestFile_a && cat /vagrant_b/SomeTestFile_b'
...
Hello world ... a
Hello world ... b
...

# destroy environment
$ vagrant destroy -f

😉

Vagrant tipps and tricks

This time a few things which make life easier.

Check for Windows

There a quit some situations for Vagrant where you have platform specific steps to do. Here an example for Windows.

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  # some content

  if Vagrant::Util::Platform.windows? then
    # do something Windows specific
  else
    # do something not Windows specific
  end
end

Set a default provider

By default, VirtualBox is the default provider for Vagrant but sometimes it is needed to change.

# set provider via vagrant up command
$ vagrant up --provider vmware_fusion

It is possible to use environment variables in Vagrantfile. So the 2nd option is to set provider inside Vagrantfile!

# -*- mode: ruby -*-
# vi: set ft=ruby :

ENV['VAGRANT_DEFAULT_PROVIDER'] = 'vmware_fusion'

Vagrant.configure("2") do |config|
  # some content
end

Multiple Vagrantfiles in one directory

Sometimes it could happen that you have multiple Vagrantfiles in one directory. In such case environment variables helps.

# select specific Vagrantfile
$ VAGRANT_VAGRANTFILE=Vagrantfile_01 vagrant up

Create log files

To enable detailed logging use the VAGRANT_LOG environmental variable.

# run with info log level (Linux and Mac OS)
$ VAGRANT_LOG=info vagrant up

# run with info log level (Windows)
$ set VAGRANT_LOG=info
$ vagrant up

Level names can be “debug”, “info”, “warn” and “error”.

Jenkins log without colored output

For Jenkins log, the color output is superfluous! Here an simple example:

pipeline {
  agent any

  stages {
    stage('Build') {
      steps {
        // Make the output without color
        vagrant up --no-color
      }
    }
  }
}

Vagrant Manager

Do you like Vagrant? … Yes? … Then you will love Vagrant Manger! Vagrant Manager is free and available for Mac OS and Windows. But pictures say more than 1000 words…

Windows

windows demo

Copyright © by Lanayo Technologies

Mac OS

mac os demo

Copyright © by Lanayo Technologies

Downloads are available via GitHub (Windows – Mac OS). The developers of Lanayo have done a very good job here!