Grafana 5.x Running on Raspberry Pi

Got around to upgrading my Raspberry Pi 3 Model B weather station with a newer version of Grafana, the Debian packages for ARM are hosted on Grafana’s download page under ARMv7.

Moving from a 2.x word to a 5.x has been impressive, most notably the drag, drop, and resize functionality.

I’m still using InfluxDB and Telegraf to store and populate my time data points.

Latest Raspbian’s (stretch) comes with Influx 1.0.2 and works nicely:

# dpkg -l | grep influxdb
ii  influxdb     1.0.2+dfsg1-1 armhf

While I needed to grab the Telegraf as a Linux Binary (ARM) from Influx’s download page.

Enjoy!

File not found!

I’ve seen it before, a customer deletes a file and then needs it restored.

Normally a challenging request, but under special circumstances a process may have the file opened.

While showing my son some fun and exciting Linux security scenarios, I recalled all those times I was able to recover data from the /proc (in memory) filesystem. In order to visualize this scenario I threw together a small Dockerfile and had him poke around:

Once logged in you notice a server.py script, for our scenario this is our companies server daemon. If we browse the code we notice a file named password.txt was opened, but never closed.

Ignore the sleep() function, this is just here to keep the file opened, and to simulate a daemon process.

The next things you should do it verify the server is actively running! If we hope to recover that file the server needs to have a lock on password.txt

root@4df66824ad14:~# ps aux
USER  PID %CPU %MEM VSZ   RSS  TTY    STAT START   TIME COMMAND
root   1  0.0  0.0  18504 3396 pts/0  Ss   01:37   0:00 /bin/bash
root  10  0.0  0.1  22548 6076 pts/0  S    01:37   0:00 python server.py
root  24  0.0  0.0  34396 2904 pts/0  R+   01:45   0:00 ps aux

We are in luck!

From here we can use the lsof command to verify password.txt is still opened:

root@4df66824ad14:~# lsof | grep password.txt
python 10 root 3r REG 8,1 30 15860072 /root/password.txt (deleted)

Bingo! we see the file is opened, and it is marked for deletion. Good thing our service is still running!

All that is left is to track down our service’s process id (we see above it’s process id, or PID is 10).

We now have all the pieces to track down our password file from the in memory filesystem.

The /proc filesystem has a number of useful files, it also includes a directory for every running process (named after the PID).

Within the process directory we have a slew of files; however, the directory we care about is fd (standing for file descriptor):

root@4df66824ad14:~# cd /proc/10
root@4df66824ad14:/proc/10# ls -l fd
total 0
lr-x------ 1 root root 64 Sep 28 01:43 0 -> /dev/null
lrwx------ 1 root root 64 Sep 28 01:43 1 -> /dev/pts/0
lrwx------ 1 root root 64 Sep 28 01:43 2 -> /dev/pts/0
lr-x------ 1 root root 64 Sep 28 01:43 3 -> '/root/password.txt (deleted)'

And just like that we found a file representing our in memory password.txt file!

From here we can show its contents, or even restore it using cp:

root@4df66824ad14:/proc/10# cat fd/3
the-most-secret-password-ever

Happy hacking!

Remote Controlled Car using Raspberry Pi and Webcam

Demo

Setup

First thing I tackled was setting up the L293D H-Bridge on the Bread Board.

I found myself referencing the following Diagram a couple times.

Step one is connecting your chip down the center of your board:

From here I connected the 3 power pins to my board’s power rail using a few Jumpers:

A few more Jumpers connect each side of the chip to ground:

Finally I use a couple Wires to connect both sides of my power and ground rails:

Using a little bit of double sided tape I stick my board onto the Car Chassis:

I also wired up each DC motor, and the battery pack to the board:

Next I wired up my Raspberry Pi‘s GPIO pins, connecting them to the L293D.

Once I’ve verified the GPIOs were connected properly, I used a couple rubber bands to strap the Pi, Portable USB Charger (I used a Vans Shoe Charger) and Web Cam to the cassis:

Code

On the Raspberry Pi I’m using Raspbian as the operating system, and installed a couple pieces of software:

* nginx
* Flask
mjpg-streamer
* Rpi.GPIO

My nginx configuration is really basic and looks like this:

server {
 listen 80 default_server;
 listen [::]:80 default_server;

 root /var/www/html;

 location /stream {
   proxy_pass http://localhost:8080/?action=stream;
   proxy_set_header Content-Type "image/jpeg";
 }

 location / {
   proxy_pass http://localhost:8000;
 }

}

The Flask application is a bit janky, but gets the job done:

app.py

import RPi.GPIO as GPIO
from flask import Flask, render_template
from flask import request

GPIO.setmode(GPIO.BCM)

GPIO.setup(14, GPIO.OUT)
GPIO.setup(15, GPIO.OUT)

GPIO.setup(23, GPIO.OUT)
GPIO.setup(24, GPIO.OUT)


app = Flask(__name__)

@app.route("/")
def index():
    return render_template('index.html')

@app.route("/left")
def left():
    method = request.args.get('method')
    if method == 'stop':
        sig = GPIO.LOW
    else:
        sig = GPIO.HIGH

    GPIO.output(23, sig)
    return "OK"

@app.route("/forward")
def forward():
    method = request.args.get('method')
    if method == 'stop':
        sig = GPIO.LOW
    else:
        sig = GPIO.HIGH

    GPIO.output(15, sig)
    GPIO.output(24, sig)
    return "OK"

@app.route("/backward")
def backward():
    method = request.args.get('method')
    if method == 'stop':
        sig = GPIO.LOW
    else:
        sig = GPIO.HIGH

    GPIO.output(14, sig)
    GPIO.output(23, sig)
    return "OK"

@app.route("/right")
def right():
    method = request.args.get('method')
    if method == 'stop':
        sig = GPIO.LOW
    else:
        sig = GPIO.HIGH

    GPIO.output(14, sig)
    return "OK"

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=8000, debug=True)

While the template contains a bit of Javascript to handle button presses:

templates/index.html

<html>

<head>

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">

<style>
  img {
      margin-top: 25px;
      margin-bottom: 25px;
    }
</style>

<script
  src="https://code.jquery.com/jquery-3.2.1.min.js"
  integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4="
  crossorigin="anonymous"></script>

<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>


<script>


// define our fired states to false
var forward_fired = false;
var backward_fired = false;
var left_fired = false;
var right_fired = false;

// keydown event will start motor
document.onkeydown = function() {

    if(event.keyCode == 87) {

        if(!forward_fired && !backward_fired) {
            forward_fired = true;

            button = document.getElementById('up');
            button.className = 'btn btn-success btn-lg disabled';

            console.log('start forward');
            $.get("/forward")
        }
    }

    if(event.keyCode == 83) {
        if(!backward_fired && !forward_fired) {
            backward_fired = true;

            button = document.getElementById('down');
            button.className = 'btn btn-success btn-lg disabled';

            console.log('start backward');
            $.get("/backward")
        }
    }

    if(event.keyCode == 65) {
        if(!left_fired && !right_fired && !backward_fired) {
            left_fired = true;

            button = document.getElementById('left');
            button.className = 'btn btn-success btn-lg disabled';

            console.log('start left');
            $.get("/left")
        }
    }

    if(event.keyCode == 68) {
        if(!right_fired && !left_fired && !backward_fired) {
            right_fired = true;

            button = document.getElementById('right');
            button.className = 'btn btn-success btn-lg disabled';

            console.log('start right');
            $.get("/right")
        }
    }

};

// keyup event will stop motor
document.onkeyup = function() {


    if(event.keyCode == 32) {
        console.log('beep');
        $.get("/beep")
    }


    if(event.keyCode == 87) {
      if(forward_fired) {
        forward_fired = false;

        button = document.getElementById('up');
        button.className = 'btn btn-default btn-lg disabled';

        console.log('stop forward');
        $.get("/forward?method=stop")
      }
    }

    if(event.keyCode == 83) {
      if(backward_fired) {
        backward_fired = false;

        button = document.getElementById('down');
        button.className = 'btn btn-default btn-lg disabled';

        console.log('stop backward');
        $.get("/backward?method=stop")
      }
    }

    if(event.keyCode == 65) {
      if(left_fired) {
        left_fired = false;

        button = document.getElementById('left');
        button.className = 'btn btn-default btn-lg disabled';

        console.log('stop left');
        $.get("/left?method=stop")
      }
    }

    if(event.keyCode == 68) {
      if(right_fired) {
        right_fired = false;

        button = document.getElementById('right');
        button.className = 'btn btn-default btn-lg disabled';

        console.log('stop right');
        $.get("/right?method=stop")
      }
    }

};

</script>

</head>

</body>


<div class="container">
  <div class="row">
    <center>

      <img src="/stream" class="img-thumbnail">

    </center>
  </div>
</div>

<!-- control buttons -->
<div class="container">
  <div class="row">

    <center>

      <button id="left" type="button" class="btn btn-default btn-lg disabled">
        <span class="glyphicon glyphicon-arrow-left" aria-hidden="true"></span>
      </button>

      <button id="up" type="button" class="btn btn-default btn-lg disabled">
        <span class="glyphicon glyphicon-arrow-up" aria-hidden="true"></span>
      </button>

      <button id="down" type="button" class="btn btn-default btn-lg disabled">
        <span class="glyphicon glyphicon-arrow-down" aria-hidden="true"></span>
      </button>

      <button id="right" type="button" class="btn btn-default btn-lg disabled">
        <span class="glyphicon glyphicon-arrow-right" aria-hidden="true"></span>
      </button>

    </center>

  </div>
</div>

</body>

</html>

I haven’t yet configured the Flask and mjpg-streamer processes to startup automatically, so I connect via a shell and start each in a screen session.

Once that is done connect to the running nginx server using your browser, you should be presented with a user interface similar to this:

Use your keyboard to control the car, just like a video game the key W moves forward, S backwards, A left, and D is right.

Component List

Here is a list of each of the components (or comparable) I used during the setup:

Raspberry Pi 3 Model B Motherboard

Official Raspberry Pi 3 Case – Red/White

uxcell White 8.5 x 5.5cm 400 Tie Points 400 Holes Solderless Breadboard

Microsoft LifeCam HD-3000 Webcam – Black (T3H-00011), 720p HD 16:9 Video Chat, Skype Certified

INSMA Motor Smart Robot Car Chassis Kit Speed Encoder Battery Box For Arduino DIY

Adafruit Dual H-Bridge Motor Driver for DC or Steppers – 600mA – L293D [ADA807]

Kuman 120pcs Breadboard Jumper Wires for Arduino Raspberry Pi 3 40pin Male to Female, 40pin Male to Male, 40pin Female to Female Ribbon Cables Kit Multicolored Pack K45

Makerfocus 140pcs Breadboard Board Jumper Cable Wire Kit w Box

Happy Will 200 PCS Breadboard Jumper Wires/Jump Wire Mix Long and Short M/M for Circuit Board

Anker Astro E1 5200mAh Candy bar-Sized Ultra Compact Portable Charger (External Battery Power Bank) with High-Speed Charging PowerIQ Technology (Black)

RaspberryPi 3 and SensorTag

I’ve been in a hackey type of mood these last couple of days. Picking up a new Raspberry Pi 3 probably did that, up until now I’ve only had the Raspberry Pi Model B.

The new features that come with the Pi 3 are great. What do I have to say, you had me at build in wifi, bluetooth and 1 GB of memory.

To take full advantage of the bluetooth I decided to learn a little about BLE (Bluetooth low energy). I began poking at my Fitbit Charge HR, but wasn’t able to do anything more than pull the device name. This led me to find the TI SensorTag. This device packs a number of sensors into a tiny package, and offers wireless communication using BLE!

screen-shot-2016-09-08-at-6-55-50-pm

After getting my Pi 3 setup with Raspbian, I installed a few bluetooth packages from the official repositories:

pi@raspberrypi:~ $ dpkg -l | grep blue
ii blueman 1.99~alpha1-1+deb8u1 armhf Graphical bluetooth manager
ii bluez 5.23-2+rpi2 armhf Bluetooth tools and daemons
ii bluez-firmware 1.2-3+rpi1 all Firmware for Bluetooth devices
ii libbluetooth3:armhf 5.23-2+rpi2 armhf Library to use the BlueZ Linux Bluetooth stack
ii pi-bluetooth 0.1.1 armhf Raspberry Pi 3 bluetooth

Using the hcitool tool provided by bluez, I scanned the area for BLE devices and found the Sensor Tag (and it’s Hardware Address).

This tool can take either lescan or scan as a parameter, where the latter will search for traditional bluetooth devices.

pi@raspberrypi:~ $ sudo hcitool lescan
LE Scan ...
A0:XX:XX:XX:XX:XX CC2650 SensorTag

Bluez also provides us gatttool, this tool provides a way to connect to the SensorTag in interactive mode (the -I flag).

Gatttool allows us to communicate with a BLE devices using the GATT protocol.

pi@raspberrypi:~ $ sudo gatttool -b A0:XX:XX:XX:XX:XX -I
[A0:XX:XX:XX:XX:XX][LE]> connect
Attempting to connect to A0:XX:XX:XX:XX:XX
Connection successful

Gatttool provides a number of useful commands, check them all out using the help command:

[A0:XX:XX:XX:XX:XX][LE]> help
help Show this help
exit Exit interactive mode
quit Exit interactive mode
connect [address [address type]] Connect to a remote device
disconnect Disconnect from a remote device
primary [UUID] Primary Service Discovery
included [start hnd [end hnd]] Find Included Services
characteristics [start hnd [end hnd [UUID]]] Characteristics Discovery
char-desc [start hnd] [end hnd] Characteristics Descriptor Discovery
char-read-hnd <handle> Characteristics Value/Descriptor Read by handle
char-read-uuid <UUID> [start hnd] [end hnd] Characteristics Value/Descriptor Read by UUID
char-write-req <handle> <new value> Characteristic Value Write (Write Request)
char-write-cmd <handle> <new value> Characteristic Value Write (No response)
sec-level [low | medium | high] Set security level. Default: low
mtu <value> Exchange MTU for GATT/ATT

The one that I’m the most interested in is char-desc, this command works as discovery, and will give us every handle on the device:

[A0:XX:XX:XX:XX:XX][LE]> char-desc
handle: 0x0001, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0002, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb
handle: 0x0004, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb
handle: 0x0006, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0007, uuid: 00002a04-0000-1000-8000-00805f9b34fb
handle: 0x0008, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0009, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x000a, uuid: 00002a05-0000-1000-8000-00805f9b34fb
handle: 0x000b, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x000c, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x000d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x000e, uuid: 00002a23-0000-1000-8000-00805f9b34fb
handle: 0x000f, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0010, uuid: 00002a24-0000-1000-8000-00805f9b34fb
handle: 0x0011, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0012, uuid: 00002a25-0000-1000-8000-00805f9b34fb
handle: 0x0013, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0014, uuid: 00002a26-0000-1000-8000-00805f9b34fb
handle: 0x0015, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0016, uuid: 00002a27-0000-1000-8000-00805f9b34fb
handle: 0x0017, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0018, uuid: 00002a28-0000-1000-8000-00805f9b34fb
handle: 0x0019, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x001a, uuid: 00002a29-0000-1000-8000-00805f9b34fb
handle: 0x001b, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x001c, uuid: 00002a2a-0000-1000-8000-00805f9b34fb
handle: 0x001d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x001e, uuid: 00002a50-0000-1000-8000-00805f9b34fb
handle: 0x001f, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0020, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0021, uuid: f000aa01-0451-4000-b000-000000000000
handle: 0x0022, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0023, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0024, uuid: f000aa02-0451-4000-b000-000000000000
handle: 0x0025, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0026, uuid: f000aa03-0451-4000-b000-000000000000
handle: 0x0027, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0028, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0029, uuid: f000aa21-0451-4000-b000-000000000000
handle: 0x002a, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x002b, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x002c, uuid: f000aa22-0451-4000-b000-000000000000
handle: 0x002d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x002e, uuid: f000aa23-0451-4000-b000-000000000000
handle: 0x002f, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0030, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0031, uuid: f000aa41-0451-4000-b000-000000000000
handle: 0x0032, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0033, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0034, uuid: f000aa42-0451-4000-b000-000000000000
handle: 0x0035, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0036, uuid: f000aa44-0451-4000-b000-000000000000
handle: 0x0037, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0038, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0039, uuid: f000aa81-0451-4000-b000-000000000000
handle: 0x003a, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x003b, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x003c, uuid: f000aa82-0451-4000-b000-000000000000
handle: 0x003d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x003e, uuid: f000aa83-0451-4000-b000-000000000000
handle: 0x003f, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0040, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0041, uuid: f000aa71-0451-4000-b000-000000000000
handle: 0x0042, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0043, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0044, uuid: f000aa72-0451-4000-b000-000000000000
handle: 0x0045, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0046, uuid: f000aa73-0451-4000-b000-000000000000
handle: 0x0047, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0048, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0049, uuid: 0000ffe1-0000-1000-8000-00805f9b34fb
handle: 0x004a, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x004b, uuid: 00002901-0000-1000-8000-00805f9b34fb
handle: 0x004c, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x004d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x004e, uuid: f000aa65-0451-4000-b000-000000000000
handle: 0x004f, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0050, uuid: f000aa66-0451-4000-b000-000000000000
handle: 0x0051, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0052, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0053, uuid: f000ac01-0451-4000-b000-000000000000
handle: 0x0054, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0055, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0056, uuid: f000ac02-0451-4000-b000-000000000000
handle: 0x0057, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0058, uuid: f000ac03-0451-4000-b000-000000000000
handle: 0x0059, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x005a, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x005b, uuid: f000ccc1-0451-4000-b000-000000000000
handle: 0x005c, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x005d, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x005e, uuid: f000ccc2-0451-4000-b000-000000000000
handle: 0x005f, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0060, uuid: f000ccc3-0451-4000-b000-000000000000
handle: 0x0061, uuid: 00002800-0000-1000-8000-00805f9b34fb
handle: 0x0062, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0063, uuid: f000ffc1-0451-4000-b000-000000000000
handle: 0x0064, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0065, uuid: 00002901-0000-1000-8000-00805f9b34fb
handle: 0x0066, uuid: 00002803-0000-1000-8000-00805f9b34fb
handle: 0x0067, uuid: f000ffc2-0451-4000-b000-000000000000
handle: 0x0068, uuid: 00002902-0000-1000-8000-00805f9b34fb
handle: 0x0069, uuid: 00002901-0000-1000-8000-00805f9b34fb

Oh boy, that is a lot of text! Luckily Texas Instruments provides a full GATT Table on the devices TearDown page.

To start out easy, I picked a handle that looked descriptive and straight forward, the Device Name!

Handle Type Permissions
0x3 Device Name R

Back in gatttool I’m going to use the char-read-hnd command to read from this handle:

[A0:XX:XX:XX:XX:XX][LE]> char-read-hnd 0x3
Characteristic value/descriptor: 53 65 6e 73 6f 72 54 61 67 20 32 2e 30

It is also worth noting we can use the longer hex value (0x0003) returned from char-desc to get the same value:

[A0:XX:XX:XX:XX:XX][LE]> char-read-hnd 0x0003
Characteristic value/descriptor: 53 65 6e 73 6f 72 54 61 67 20 32 2e 30

The value returned is a set of hexadecimal values. Using Python I can  decode them and learn their secrets 😛

In [1]: val = '53 65 6e 73 6f 72 54 61 67 20 32 2e 30'

In [2]: [ i.decode('hex') for i in val.split() ]
Out[2]: ['S', 'e', 'n', 's', 'o', 'r', 'T', 'a', 'g', ' ', '2', '.', '0']

In [3]: ''.join([ i.decode('hex') for i in val.split() ])
Out[3]: 'SensorTag 2.0'

Check back soon, I plan to post more as I learn about this sweet device.

Process Elasticsearch JSON on the shell

Lets throw security out the window for a moment. Say we store user accounts with clear text passwords in Elasticsearch, what is the easiest way to use the results in a shell script? We can begin by creating two accounts, one for admin and one for john:

# curl -XPUT localhost:9200/site/people/1?pretty=True -d '
  {"name": "admin", "password": "secret", "admin": "true"}
'
{
  "_index" : "site",
  "_type" : "people",
  "_id" : "1",
  "_version" : 1,
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
 "created" : true
}
# curl -XPUT localhost:9200/site/people/2?pretty=True -d '
  {"name": "john", "password": "password", "admin": "false"}
'
{
  "_index" : "site",
  "_type" : "people",
  "_id" : "1",
  "_version" : 2,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
 "created" : false
}

Using curl this is very easy to query, we just use the id:

# curl localhost:9200/site/people/2?pretty=True
{
  "_index" : "site",
  "_type" : "people",
  "_id" : "2",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "name" : "john",
    "password" : "password",
    "admin" : "false"
  }
}

But what we really want is to get the values of the keys without needing to hack it with sed/awk/grep. We can install a small json parsing package called jq:

# apt-get install jq

Now we can easily pull the keys out:

# curl -s localhost:9200/site/people/2 | jq ._source.name
"john"

# curl -s localhost:9200/site/people/2 | jq ._source.password
"password"

# curl -s localhost:9200/site/people/2 | jq ._source.admin
"false"

Using the power of jq we could easily search Elasticsearch and perform comparisons for login, remember we are throwing security out the window here, this is merely for example:

# curl -s $ip:9200/site/people/_search?q=name:admin | jq -r '.hits.hits'
[
  {
    "_source": {
      "admin": "true",
      "password": "secret",
      "name": "admin"
    },
    "_score": 0.30685282,
    "_id": "1",
    "_type": "people",
    "_index": "site"
  }
]

We can assume this query only returns one user, and we are pretty sure the account exists, so lets grab the password key directly:

# curl -s $ip:9200/site/people/_search?q=name:admin |
  jq -r '.hits.hits[0]._source.password'
secret

If the value is not present, and we attempt to access it, we will get a null result:

# curl -s $ip:9200/site/people/_search?q=name:jack |
  jq -r '.hits.hits[0]._source.password'

null

And here it is all wrapped up in a poorly written shell script, I would not advise using this logic for anything more than toying with:

auth.sh

#!/bin/bash

read -p "Login username: " username
read -p "Login password: " password

account=`curl -s localhost:9200/site/people/_search?q=name:$username |
  jq '.hits.hits[0]'`

if [ "$account" != "null" ] ; then

  account_username=`echo $account | jq -r ._source.name`

  if [ "$username" == "$account_username" ] ; then

    account_password=`echo $account | jq -r ._source.password`

    if [ "$password" == "$account_password" ] ; then
      echo "You are in!"
    else
      echo "Wrong username or password"
    fi

  fi

fi

And usage looks like this:

# ./auth.sh
Login username: admin
Login password: password
Wrong username or password
# ./auth.sh
Login username: john
Login password: password
You are in!