Version control for scientific microcontrollers with git hooks

In our lab we are using microcontrollers (mainly arduinos) for a variety of tasks. These tasks can be as simple as forwarding a serial command to BNC to trigger a stimulation device, or waiting for button presses to measure reaction time. Often these tasks depend on settings (e.g. a threshold for analog-read, pulse-width in microseconds, or which BNC out to trigger).

It is impractical to hard-code such settings into the firmware. By using a protocol for the serial interface to the arduino, one can change settings at runtime. Furthermore, this allows to request the current settings, and receive them from the device..

Especially in a scientific environment, where we do rapid development and source-code changes often, one can easily lose track of the current version installed. Being able to requesting information about the current version of firmware can therefore be quite practical.

Semantic versioning and compile-time

Defining a variable using semantic versioning (e.g. const String versionInfo = "'v0.0.1 - EdgyExample'";) or the __TIME__ and __DATE___ macros allow us do aggregate information about the firmware and construct a json-string. Then, whenever we send the enquiry to the device (we like to use ASCII 0x05), it answers with this string (see the arduino-code). From the PC side, we can then request the current version, and parse the received byte stream (see python-code). This approach is probably already feasible and sufficient for most use cases, especially if we keep a linear chain of commits.

There are some gaps though. First, there is no clear link between the version defined in in the string and the version checked out. Git tags have to be kept manually in line with the string. Additionally, compile-time and date only indicate the time of the last compile, not the actual version which has been checked out.

Hooking the Hash

That means that if we roll back to an earlier version, __TIME__ and __DATE__ are not as informative anymore. But if we keep our source-code under version control, say git, why not just use the information from there? Is there a way to put the hash of the current commit into the source-code before it is compiled and downloaded?

Obviously, the json-string could be expanded, and we can included a field for the hash. This could be achieved by creating a file, say hash.h which defines const char * hash = "9a99cd8..."; and including it with #include hash.h. But we would have to create and fill the hash.h file.

Ideally this should be be done automatically. Git hooks to the rescue!

The first step is writing a script that creates such a file. Git hooks reside under .git/hooks. The next step is therefore calling this script from .git/hooks/post-commit and .git/hooks/post-checkout. This can be done by just replacing those files with the script. But consider that hooks are themselves not under version control. Because we develop on multiple computers, this can be impractical. Therefore, i like to create a folder hooks in the root directory of the repository, which contains the shell script called create-hash that actually creates the hash, two shell-scripts called post-commit and post-checkout which are the prototypes for the hooks, and an install-script that just copies the hook-scripts to .git/hooks. This allows me to keep the hooks under version control, and only requires to install once on each computer.

Conclusion

This is not bullet-proof though, as the hooks are only executed after you commit or checkout. That means if you hack away on your code, compile and download it, without committing it first, it still uses the old hash. Which makes total sense - as the new one was not created yet. Additionally, i like to .gitignore the hash-file, as every commit and checkout alters the files.

Code on the arduino

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

#include "hash.h" //defines HASH 
const String versionInfo = "\"v0.0.1 - EdgyExample\"";
byte serialBuffer      = 0x00;
byte serialCommand     = 0x00;

void setup(){
    Serial.begin(115200);
}

void loop(){
  if (Serial.available() > 0) {
    // read the incoming bytes from serial:
    serialCommand    = byte(Serial.read());
    // throw away the rest until you receive a 0x0a, i.e. newline: '\n'
    while (serialBuffer != 0x0a) {serialBuffer  = byte(Serial.read());}
    serialBuffer = 0x00;

    // execute command
    if (serialCommand == 0x1b) {setup();} // reset
    else if (serialCommand == 0x05) {send_enquiry_response();} //enquire
  }
}

inline void send_enquiry_response() {
  Serial.print("{\"version\":");
  Serial.print(versionInfo);
  Serial.print(", \"compile-date\":\"");
  Serial.print(__DATE__);
  Serial.print("\", \"compile-time\":\"");
  Serial.print(__TIME__);
  Serial.print("\", \"git-hash\":\"");
  Serial.print(HASH);
  Serial.println("\"}");
  Serial.flush();
}

Python interface

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

#!/usr/bin/env python3
import serial
from serial.tools.list_ports import comports
import json
import struct
from time import sleep


def available(port: str=None):
    availablePorts = [a.device for a in comports() if
                      a.vid == 6790 and a.pid == 29987]
    # these are arduino vendor and product ids

    if len(availablePorts) == 0:
        raise ConnectionError("No device connected at any port")

    if port is None:            
        return availablePorts

    if port.upper() in availablePorts:
        return [port]
    else:
        raise ConnectionError("No device connected at " + port)

class Arduino():
    def __init__(self,
                 port=None,
                 baud=115200,
                 ):
        print("Connecting arduino")
        self.port = available(port)[0]
        self.baud = baud
        self.interface = serial.Serial(port=self.port, baudrate=self.baud)
        sleep(5)

    def write(self, message: bytes):
        for l in serial.iterbytes(message):
            self.interface.write(l)
        self.interface.flush()

    def enquire(self, verbose=False):
        msg = b'\x05\n'
        responses = self.query(msg)
        return responses
    
    def receive(self, blocking=False):
        buffer = b''
        while buffer[-2:] != b'\r\n':
            message = self.interface.read_all()
            buffer += message

        lines = buffer.splitlines()
        dicts = []
        for l in lines:
            try:
                decoded = l.decode()
                d = json.loads(decoded)
                dicts.append(d)
            except json.JSONDecodeError as e:
                print(decoded)
                raise e
        return dicts

    def query(self, msg: bytes):
        self.write(msg)
        return self.receive(True)

if __name__ == "__main__":
    a = Arduino()
    replies = a.enquire()
    for reply in replies:
        print(reply)

Bash to create hash-file

can be used for post-commit and post-checkout in .git/hooks

1
2
3
4

#!/bin/bash
SHA1=$(git rev-parse --verify HEAD)
echo "const char * HASH = \""$SHA1"\";" > ./hash.h
exit 0;