Airprobe with GNU Radio 3.7

I'm very excited right now.. I ordered a HackRF and can't wait for it to be delivered to me now.

Since I heard of the HackRF project from Michael Ossmann (http://greatscottgadgets.com/hackrf/) I knew at some day I will buy one. I've started my way to SDR last year by buying a RTL-SDR stick and also did a little project with an USRP1, which I borrowed from my university.

So now I'm trying to set up my GNU Radio environment again and prepare it for the HackRF. And by doing so I've stumbled across a little problem:

Airprobe (a software to  decode GSM) wouldn't compile with the new GNU Radio version 3.7+. The problem is that GNU Radio changed the API with the 3.7 version and therefore breaking the compatibility with airprobe. Fortunately, I found out somebody has already patched airprobe to compile and run (didn't do extensive testing, since my HackRF has not arrived yet) with GNU Radio 3.7. Nevertheless, there where some difficulties and therefore I wrote this post the next day. I hope I remembered every step I did. Please write me a comment if you find mistakes or if you have problems in following the steps...

Installing GNU Radio 3.7

GNU Radio 3.7 comes with PyBombs (which is awesome). That means we don't need the build-gnuradio script anymore (you can still use it though). With PyBombs you do it like this:
(here is the detailed tutorial: http://gnuradio.org/redmine/projects/pybombs/wiki)

 $ cd /opt

 $ sudo mkdir pybombs target

 $ sudo chown dennis:dennis pybombs target

 $ git clone https://github.com/pybombs/pybombs.git
 $ cd pybombs
 $ ./pybombs install gnuradio
 $ /opt/target/setup_env.sh

Now it will ask you some questions (e.g. which install prefix to use; I use /opt/target) and then it will start installing all dependencies (first by looking for .deb packets; only if no packets where found it uses the sources) and finally download and compile GNU Radio 3.7.

Note that I don't have to run the installation as root, since the two directories '/opt/pybombs' and '/opt/target' are belonging to my user. GNU Radio will install in /opt/target and not under /usr/local.

That is also the reason for the setup_env.sh script. It sets the environment variables correctly. You will have to run this script every time you restart your machine and want to use gnu radio.

That was easy. On my system (Ubuntu 14.04) this worked without any problems (it took some hours though^^). But note that my system wasn't a 'fresh' Ubuntu, but one with all kinds of stuff already installed on it. So you might run in some errors I didn't had. Just write a commend if you stuck at this point...

By the way:

PyBombs can be used to install all kinds of stuff, just run

 /opt/pybombs$ ./app_store.py

to have a look what other modules can be installed. Some of them might not work though...

For example airprobe -.-

So we have to do that the old fashion way.

Installing libosmocore

Airprobe depends on libosmocore, so we have to install that first:

 $ cd /opt/pybombs/src
 $ git clone git://git.osmocom.org/libosmocore.git
 $ cd libosmocore/
 $ ./configure --prefix=/opt/target
 $ make
 $ make install
 $ sudo ldconfig

Installing Airprobe

When I first tried to install airprobe, I did it via the app_store. What this does is just downloading the
sources from git://svn.berlin.ccc.de/airprobe to /opt/pybombs/src/ and that's it. Unfortunately I found out, that the patch I found online, doesn't match with this version of airprobe. So if you also tried it this way, delete the airprobe directory in /opt/pybombs/src. We'll use another repository.

First we download the sources:

 $ cd /opt/pybombs/src
 $ git clone git://git.gnumonks.org/airprobe.git
 $ cd airprobe

Now we download and apply the patch from zmiana. You can find the patch on github at this link: https://github.com/scateu/airprobe-3.7-hackrf-patch. It is called zmiana.patch. A howto is also provided at the page, but you can also read on here.. Btw a big thanks to zmiana for doing all the work for us!

 /opt/pybombs/src/airprobe$ patch -p1 < zmiana.patch
 /opt/pybombs/src/airprobe$ cd gsmdecode

 /opt/pybombs/src/airprobe/gsmdecode$ ./bootstrap

 /opt/pybombs/src/airprobe/gsmdecode$ ./configure --prefix=/opt/target

 /opt/pybombs/src/airprobe/gsmdecode$ make

 /opt/pybombs/src/airprobe/gsmdecode$ cd ../gsm-receiver

 /opt/pybombs/src/airprobe/gsm-receiver$ ./bootstrap

 /opt/pybombs/src/airprobe/gsm-receiver$ ./configure --prefix=/opt/target

 /opt/pybombs/src/airprobe/gsm-receiver$ make

Now we should be able to do a quick test. Download this capture file: cfile

Also start a instance of wireshark and start listening on the loopback interface. Then we start decoding the cfile:

 $ cd src/python

 $ ./go.sh ~/Downloads/capture_941.8M_112.cfile

The result should be decoded packets flushing down the terminal and you should also be able to see them in your wireshark trace.

That's it. I didn't test anything else since I don't have my HackRF yet. However, note that on https://github.com/scateu/airprobe-3.7-hackrf-patch there is also a python program called gsm_receive_hackrf_3.7.py that hopefully enables GSM capturing with the HackRF. Somebody out there who can confirm that?

Have fun and leave a comment!

Suspend on low battery in Ubuntu GNOME 14.04

I run 14.04 since it came out in April 2014. And I love it. They improved the touch input which is awesome on my Thinkpad x220 Tablet/Convertable. There was only a little thing that drove me nuts:

The suspand action on critical battery seemed to stop working in this Ubuntu version. Instead my machine just shut down every time the battery was critical low.. without asking. That's horrible when you have documents open. Today I finally found the solution: pm-utils was missing.

But let me start from the beginning..

Beside the symptom that the laptop shuts down instead of suspending, the following things seemed a bit weird:

upower tells me that suspend is not possible:

$ upower -d

Device: /org/freedesktop/UPower/devices/line_power_AC

... < I stripped of the uninteresting outputs. Here is what matters: >

Daemon:

  daemon-version:  0.9.23

  can-suspend:     no

  can-hibernate:   no

  on-battery:      yes

  on-low-battery:  no

  lid-is-closed:   no

  lid-is-present:  yes

  is-docked:       no

Don't get me wrong: Suspending is working when I just click on the suspend button. So I tried it via a dbus command:

$ dbus-send --system --print-reply --dest="org.freedesktop.UPower" /org/freedesktop/UPower org.freedesktop.UPower.Suspend

it outputs:

Error org.freedesktop.UPower.GeneralError: No kernel support

Now thats really weird. Than I found this on the internet: https://bbs.archlinux.org/viewtopic.php?id=147272&p=7

There they guess that upower still uses 'pm-is-supported' from the pm-utils package. Even though the upower package doesn't have this package as a dependency. So I installed it:

$ sudo apt-get install pm-utils

$ sudo reboot

And voilà:  upower is saying that I'm now able to suspend and its working as critical low battery action as well. I hope this will help somebody out there who has the same problem ;)

Measuring the Latency of a Linux Bridge

As you can see I am working at the moment with a Linux Bridge and try to figure out how good it does its job ;) Therefore I had to measure the latency (or simply the time) a packet needs to go from one port of the bridge (IN) to the other (OUT). My test setup looks like this:

To measure the latency I just have to send a single packet from my test machine's first Ethernet interface and wait until it arrives at the other one. So I started two instances of tcpdump to listen on both interfaces and compared the timestamps when they catch the packet. It's pretty straight forward:

First timestamp (eth0): 1396835791.679071s
Second timestamp (eth1): 1396835791.679389s
Latency is 1396835791.679389s - 1396835791.679071s = 318 microseconds

Notice that you have to start tcpdump with the option "-tt" to get this format of timestamps. If you just want the time the packet spends in the bridging device, you just have to do this measurement a second time but without the device (eth0 and eth1 short-circuit). Then you can estimate the time spend in the device by again subtracting your 2 measurements.

To get meaningful results you definitely want to do this procedure more than once and calculate the average. And then you want to vary the size of the packet, the protocol and so on to see how it's effecting the result. So at this point I decided to automate the whole procedure with a shell script. This has the big advantage that you can do this measurement again in the exact same way after doing some optimization on the bridge.

As always I started with a simple script but after tweaking this and that it got bigger and way more complex xD Nevertheless I want to share it, so maybe someone will find it useful too. Tools needed are of course tcpdump to capture the packets, as well as hping3 and arping to generate the packets.

I split the script into two: One that does exactly one measurement and another that calls this script in a loop to get the average values.

measureLatency.sh:

#!/bin/bash
#
# Simple script to send an arbitrary packet via an interface
# and measure how long it takes til it arrives on another
# interface (so both interfaces have to be connected to the same
# subnet. Of course you want to put a bridge or something
# similar in between to test its performance!
#
# Dennis Mantz <dennis.mantz@googlemail.com> - 2014 Lantzville

# function to translate the timestamp into microseconds:
function tstampToMicro()
{
# Don't forget to remove leading zeros or you do octa ^^
seconds=`echo $1 | cut -d "." -f 1 | sed 's/^0*//'`
microseconds=`echo $1 | cut -d "." -f 2 | sed 's/^0*//'`

echo $(($seconds * 1000000 + $microseconds))
}

# Function to show the usage:
function show_usage()
{
echo "Usage: $cmdname -s <src iface> -d <dst iface> [-m <arp|ip|icmp|udp|tcp>] [-p <size>] [-v] [-h]"
exit -1
}

# Set default values:
cmdname=$0
ifs=
ifd=
size=0
mode="tcp"
verbose=
arpingVerbose="-q"
hping3Verbose="-q"

# Parse cmd line args:
while getopts "s:d:m:p:vhc:" opt; do
case "$opt" in
s)
ifs=$OPTARG
;;
d)
ifd=$OPTARG
;;
m)
mode=$OPTARG
;;
v)
verbose="-v"
arpingVerbose=
hping3Verbose="-V"
;;
h)
show_usage
;;
p)
size=$OPTARG
;;
c)
# This is just for compatibility with the averageLatency script. Do nothing with it...
;;
esac
done

# Check args:
if [ -z "$ifs" -o -z "$ifd" ]
  then
    show_usage
fi

# Check root
if [ `whoami` != "root" ]
  then
    echo "Must be root!"
    exit -1
fi

# Parse mode
case $mode in
arp)
filter='arp'
;;
ip)
filter='ip'
;;
icmp)
filter='icmp'
;;
tcp)
filter='tcp port 5000'
;;
udp)
filter='udp port 5000'
;;
*)
echo "$mode is not allowed for mode"
show_usage
;;
esac

# Start tcpdump two times:
(tcpdump -tt -i $ifs -c 1 $filter 2> /tmp/measureLatencyTcpdump1.err > /tmp/measureLatencyCap1.tmp)&
pid1=$!
(tcpdump -tt -i $ifd -c 1 $filter 2> /tmp/measureLatencyTcpdump2.err > /tmp/measureLatencyCap2.tmp)&
pid2=$!

# wait for them to get ready:
sleep 3

# Send a packet on the first interface:
case $mode in
        arp)
                arping -I $ifs -c 1 111.111.111.111 -s 127.0.0.1 $arpingVerbose
                ;;
        ip)
                ifconfig $ifs 111.111.111.1 netmask 255.255.255.0 broadcast 111.111.111.254 # add a route
                arp -i $ifs -s 111.111.111.111 00:01:11:11:11:11 $verbose # dummy arp entry..
                hping3 --rawip 111.111.111.111 -H 99 -c 1 -d $size $hping3Verbose > /tmp/measureLatencyHPing3.out 2> /tmp/measureLatencyHPing3.err
                ;;
icmp)
ifconfig $ifs 111.111.111.1 netmask 255.255.255.0 broadcast 111.111.111.254 # add a route
                arp -i $ifs -s 111.111.111.111 00:01:11:11:11:11 $verbose # dummy arp entry..
                hping3 --icmp 111.111.111.111 -c 1 -d $size $hping3Verbose > /tmp/measureLatencyHPing3.out 2> /tmp/measureLatencyHPing3.err
                ;;
        tcp)
                ifconfig $ifs 111.111.111.1 netmask 255.255.255.0 broadcast 111.111.111.254 # add a route
                arp -i $ifs -s 111.111.111.111 00:01:11:11:11:11 $verbose # dummy arp entry..
                hping3 -s 5000 111.111.111.111 -c 1 -d $size $hping3Verbose > /tmp/measureLatencyHPing3.out 2> /tmp/measureLatencyHPing3.err
                ;;
        udp)
                ifconfig $ifs 111.111.111.1 netmask 255.255.255.0 broadcast 111.111.111.254 # add a route
                arp -i $ifs -s 111.111.111.111 00:01:11:11:11:11 $verbose # dummy arp entry..
                hping3 --udp -s 5000 111.111.111.111 -c 1 -d $size $hping3Verbose > /tmp/measureLatencyHPing3.out 2> /tmp/measureLatencyHPing3.err
                ;;
esac

# wait for both tcpdumps to finish:
(sleep 10; kill $pid1 2> /dev/null; kill $pid2 2> /dev/null; echo "Didn't receive the packet!")&
killerpid=$!
wait $pid1
wait $pid2
kill $killerpid 2> /dev/null
wait $killerpid 2> /dev/null

# get the timestamps:
time1=`cat /tmp/measureLatencyCap1.tmp | sed 's/\([0-9]*\.[0-9]*\)\(.*\)/\1/'`
time2=`cat /tmp/measureLatencyCap2.tmp | sed 's/\([0-9]*\.[0-9]*\)\(.*\)/\1/'`

if [ "a$verbose" = "a-v" ]
  then
    echo "time1 = $time1"
    echo "time2 = $time2"
    cat /tmp/measureLatency*
fi

if [ -z "$time1" -o -z "$time2" ]
  then
    echo "Timestamps aren't set. exiting..."
    exit -1
fi

# Check if it's the same packet!
pack1=`cat /tmp/measureLatencyCap1.tmp | sed 's/\([0-9]*\.[0-9]*\)\(.*\)/\2/'`
pack2=`cat /tmp/measureLatencyCap2.tmp | sed 's/\([0-9]*\.[0-9]*\)\(.*\)/\2/'`
if [ "${pack1:0:60}" != "${pack2:0:60}" ]
  then
    echo "Seems like we've captured two different packets. Is the link free?"
    exit -1
fi

# calculate the difference:
diff=$((`tstampToMicro $time2` - `tstampToMicro $time1`))
echo "latency is $diff microseconds"

# clean up:
rm /tmp/measureLatency*

avgLatency.sh:

#!/bin/bash
# Simple script to send an arbitrary packet via an interface
# and measure how long it takes in AVERAGE til it arrives on another
# interface (so both interfaces have to be connected to the same
# subnet. Of course you want to put a bridge or something
# similar in between to test its performance!
#
# Dennis Mantz <dennis.mantz@googlemail.com> - 2014 Lantzville

# Function to calculate the average:

function calc_avg() {
avg=$(($sum / $((i-1))))
range=$(($max - $min))
echo "Average Latency is $avg microseconds!"
echo "Min: $min  Max: $max  Range (max-min): $range"
}

# Signalhandler for SIGINT calls calc and exits
trap "echo 'signal causing interrupt..'; calc_avg; exit" SIGHUP SIGINT SIGTERM

# Set Vars:
sum=0
min=10000
max=0
cmdline=$*
n=5

# Parse cmd line args:
while getopts "s:d:m:p:vhc:" opt; do
        case "$opt" in
        c)
                n=$OPTARG
                ;;
*)
# All other args are passed with the $cmdline to measureLatency. Do nothing with them..
;;
        esac
done

# Run measureLatency
for i in `seq $n`
  do
echo -n "Round $i: "
latency=`./measureLatency.sh $cmdline | grep "latency" | cut -d " " -f 3`
echo "$latency us"
sum=$(($sum + $latency))
if [ $latency -lt $min ]
 then
   min=$latency
fi
if [ $latency -gt $max ]
          then
            max=$latency
        fi
  done

i=$((i+1))
calc_avg

As I said, they are a bit more complex than I wanted them to be. After I finished my measurements I also thought about optimizing this whole procedure with a c program using raw sockets. Because as you will discover by yourself: tcpdump is so damn slow when you tell it to just capture one packet and exit. A self written program would be much faster and 'nicer' :) But unfortunately until now I didn't had time to start working on it and so it goes straight do my ToDo list for the future ;)

Profiling kernel modules with oprofile

== Description ==

oprofile is a system-wide profiler which can be used to profile any user-space application as well as the kernel itself and its modules. It has the capability to produce annotated source from binaries which are compiled with debug symbols (gcc -g).

In this blog I want to describe how to set up oprofile to profile the linux ethernet bridge module.

== Used System ==

I used an Ubuntu 12.04 LTS, but the steps should be the same on other systems. In order to setup a bridge you'll need to physical network interfaces. I used the laptop's build in nic and a usb-to-nic adapter. The build in nic (eth0) is connected to the network and the usb-to-nic adapter is connected to a second laptop which connects to the network via the bridge.

== Preparations ==

In order to profile the bridge module we need to recompile the kernel with debug symbols and profiling support. We also need to compile and install oprofile itself if it's not already installed.

== Recompile the Kernel ==

Download the kernel sources for your kernel version (or a newer version if you like).
I used git (if git is not yet installed on your system use: $ sudo apt-get install git)

$ git clone git://kernel.ubuntu.com/ubuntu/ubuntu-precise.git
$ cd ubuntu-precise
$ git tag -l

With the last command we printed out a list of tags we can choose from. I chose the one which is closest to my running kernel to keep complications to a minimum:

$ git checkout -b mybranch Ubuntu-lts-3.8.0-34.49_precise1

To compile the kernel we need additional packages (I may have forgotten some)

$ sudo apt-get install build-essential binutils libncurses5-dev

Then we copy the config of our running kernel into the source root:

$ cp /boot/config-`uname -r` .config

Now we can activate the config options we need for profiling to work:

$ make menuconfig

-> Set the following options:
General setup -> Profiling support = y
General setup -> OProfile system profiling = y
Kernel hacking -> Strip assembler-generated symbols during link = n
Kernel hacking -> Compile the kernel with debug info = y
Networking support -> Networking options -> 802.1d Ethernet Bridging = m

Exit and save changes. Then we build and install the new Kernel (this will take a while).
If you have a cpu with multiple cores, you can specify -j 1+<number of cores> to speed the build up. I have a quad-core cpu, so I'll build with -j 5:

$ make -j 5
$ sudo make deb-pkg
$ cd ..
$ sudo dpkg -i linux-image-3.8.13.12_3.8.13.12-2_amd64.deb
$ sudo dpkg -i linux-headers-3.8.13.12_3.8.13.12-2_amd64.deb
$ sudo dpkg -i linux-firmware-image_3.8.13.12-2_amd64.deb
$ sudo dpkg -i linux-libc-dev_3.8.13.12-2_amd64.deb

Finally we can reboot our system and start the new kernel!

== Install and setup the Bridge ==

First we need to install the bridge utilities:

$ sudo apt-get install bridge-utils

Then we add a new logical bridge interface (br0) and add the two physical interfaces (eth0 and eth1) to it. By the way: I strongly recommend to deactivate the network manager before setting up the bridge!

$ sudo ip addr flush eth0
$ sudo ip addr flush eth1
$ sudo brctl addbr br0
$ sudo brctl addif br0 eth0 eth1
$ sudo ip link set dev br0 up

Now the bridge should be up and running. If you have a dhcp-server in your network, you can run dhclient on the bridge-interface to assign it an IP address (otherwise do a static IP configuration on the bridge):

$ sudo dhclient br0

== Install oprofile and start the profiling ==

To install oprofile we will download the sources of the newest version from the website (at the time this howto was written this was version 0.9.9) and compile them. Before we do so, we have install some more dependencies.

$ sudo apt-get install libpopt-dev binutils-dev
$ wget http://prdownloads.sourceforge.net/oprofile/oprofile-0.9.9.tar.gz
$ tar -xvf oprofile-0.9.9.tar.gz
$ cd oprofile-0.9.9
$ ./configure
$ make
$ make install

Now we're finally ready to start the profiling:

$ sudo opcontrol --init
$ sudo opcontrol --start --vmlinux=/home/user/ubuntu-precise/vmlinux

If the watchdog service is using the NMI on the machine, oprofile will exit with an error and tell you to deactivate watchdog.
If this happens do the following:

$ sudo opcontrol --deinit
$ su root
$ echo 0 > /proc/sys/kernel/nmi_watchdog
$ exit

Then do the init and start commands again. The profiling is now running in the background. In order to output the results you should run these two commands:

$ sudo opcontrol --dump   # This dumps all collected profiling data to the hard drive.
$ opreport # This generates a profiling report of the whole system.

Running the opreport command with -l will generate a more detailed report with all symbols separated. To get only the information about the binary you care about, you have to specify the name of the binary:

$ opcontrol -l /usr/bin/firefox

or in case of a kernel module (you have to give the path to the kernel modules):

$ opcontrol -l --image-path=/lib/modules/`uname -r`/kernel  /lib/modules/`uname -r`/kernel/net/bridge/bridge.ko

There is also the possibility to generate annotated source code with the opannotate command:

$ opannotate --image-path=/lib/modules/`uname -r`/kernel --output-dir=~/profiling-output

After this completes the profiling-output directory contains all source files with annotations.

== Examples and Tips for getting started ==

OProfile could be configured in many different ways, to profile exactly the things you really want to see.
The next view lines generate some interesting and basic outputs.

Callgraph
To generate a callgraph, run opcontrol and opreport with the --callgraph option:

$ sudo operf --start --callgraph --vmlinux /home/dxm02271/ubuntu-precise/vmlinux
$ opreport --callgraph /lib/modules/3.8.13.12/kernel/net/bridge/bridge.ko --merge all --image-path=/lib/modules/3.8.13.12/kernel/

This will give you a callgraph that looks like this:

---------------------------------------------------------------------
  90       100.000  bridge.ko                br_handle_frame
90       14.2631  bridge.ko                br_handle_frame
  3680     95.3121  vmlinux                  nf_hook_slow
  90        2.3310  bridge.ko                br_handle_frame
  90        2.3310  bridge.ko                br_handle_frame [self]
  1         0.0259  vmlinux                  nf_iterate
---------------------------------------------------------------------
....

Notice there's one line, that isn't intended. That's the function which is in focus. All lines above are functions calling it and all lines beneath are functions getting called by it.

I hope this was interesting for you. Please leave me a comment if ;)