Reverse engineering the Alesis V-series SysEx protocol.

I recently got back into music production and decided to order myself a MIDI controller. I got a few recommendations for the the Alesis V25, so I went ahead and ordered it. However, I was less than pleased to find that its configuration software wouldn’t run on Linux, even under Wine. Of course, this prompted me to reverse engineer the protocol that lets the software talk to the keyboard.

Overview of MIDI

Musical Instrument Digital Interface (MIDI) is a long-lived standard for interactions both among musical instruments and between instruments and software. The Alesis V25 is a MIDI controller, which is essentially just a keyboard that lets you send notes to a Digital Audio Workstation (DAW) such as Bitwig (which is just my personal favorite).

MIDI messages are always three bytes, and can look something like this (in hexadecimal):

09 30 6a

The first byte is called the “status byte,” and the second two are simply called “data bytes.”

The meaning of the particular bytes in the above message are as follows:

09 Channel 0, note on
30 Note 48 (C#4)
6a Velocity 106

In addition to note messages, there are also Control Change (CC) messages which indicate that a knob was turned or a button was pressed. However, these standard messages aren’t going to be very important for the adventure we’re going on today. Reconfiguring MIDI devices uses an extended protocol called SysEx.

Introduction to SysEx

System Exclusive (SysEx) is an extension to MIDI that lets instrument manufacturers define custom commands for their devices. The specification is quite simple: it only requires that SysEx messages start with a two byte header, and end with a one-byte trailer:

f0 xx … f7

Here, f0 is the start byte, xx is a placeholder for a manufacturer ID, and f7 is the end byte. The bytes in between can be anything, as long as their high bits aren’t set (i.e., they are less than or equal to 7f). Their meaning is interpreted on a manufacturer-by-manufacturer basis.

Overview of the Alesis V25 Editor

Alesis’ editor lets you do things such as reassign the control numbers on the keyboard’s knobs and buttons. It doesn’t do much — it mostly just supports reading the configuration from the keyboard, editing it, and writing it back. The interface looks something like this:

Since the software doesn’t work on Linux, I had it running in a virtual Windows machine under Virtualbox, with USB forwarding enabled for the device.

Intercepting SysEx Messages

We want to see what this software is doing in the backend to update the MIDI device’s configuration. To do that, we’ll need to snoop on the SysEx messages that it exchanges with the device.

It turns out that the popular network-capture software Wireshark supports snooping on USB devices. That’s exactly what we’ll need to do today.

First, I had to enable the usbmon kernel module:

sudo modprobe usbmon

Then, I started up Wireshark and started listening for packets. The display was initially quite noisy due to other USB devices (mostly my mouse). I found the bus and device IDs via lsusb and filtered on those.

In this particular capture I’ve simply queried for the controller’s current configuration and received a reply. The 80-byte packet is the query, and the 128-byte packets are chunks of the reply. The 64-byte messages in between appear to be acknowledgements occurring at the USB protocol level.

We can dissect one of the packets to see what’s going on:

We can see that the SysEx message has been divided up into 3-byte segments in accordance with the traditional MIDI specification. At the USB level, each 3-byte segment is given a header to indicate which device it is from and how many of the SysEx bytes are meaningful. The details of that aren’t important, but the result is that we are able to pull out the actual SysEx message for the query:

f0 00 00 0e 00 41 62 00 5d f7

We can identify the SysEx start and end bytes, and conveniently the manufacturer ID is 00. A bit of research has driven me toward the conclusion that because Alesis hasn’t officially registered a MIDI manufacturer ID, it uses 00 with a two-byte extension of 00 0e. I haven’t deciphered all of this message, however have been able to determine that the byte 62 is the important part; it indicates that this packet is a query for the current configuration.

Decoding the Reply

To determine which part of the SysEx messages do what, I had to play around with the GUI for a while, changing options to see how they affected the data going over the wire. This was a tedious process, however in the end it only took about three hours.

The reply is quite long, and is split over several USB packets, however I’ve copied and annotated the final SysEx message here:

Raw bytes Section Interpretation
f0 SysEx start byte
00 00 0e Alesis manufacturer ID
00 41 63 00 5d Some sort of header The 3rd byte indicates message type:

  • 61: Set configuration
  • 62: Query configuration
  • 63: Current configuration (reply)
0c 02 00 00 Keys configuration
  1. Base note
  2. Current octave
  3. Channel
  4. Velocity curve
00 Pitch wheel configuration
  1. Channel
00 01 00 7f Mod wheel configuration
  1. Channel
  2. CC number
  3. Minimum value
  4. Maximum value
40 00 7f 00 Sustain pedal configuration
  1. CC number
  2. Minimum value
  3. Maximum value
  4. Channel
00 14 00 7f 00
00 15 00 7f 00
00 16 00 7f 00
00 17 00 7f 00
 Knobs configuration
  1. Operation mode
    • 00: CC
    • 01: Aftertouch
  2. CC number
  3. Minimum value
  4. Maximum value
  5. Channel
00 31 00 00 09
00 20 00 00 09
00 2a 00 00 09
00 2e 00 00 09
00 24 00 00 09
00 25 00 00 09
00 26 00 00 09
00 27 00 00 09
 Pads configuration
  1. Operation mode
    • 00: Note
    • 01: Toggle CC
    • 02: Momentary CC
  2. Note / CC number
  3. Fixed (?) / Minimum CC value
  4. Velocity curve / Maximum CC value
  5. Channel
00 30 7f 00 00
00 31 7f 00 00
00 32 7f 00 00
00 33 7f 00 00
Buttons configuration
  1. Operation mode
    • 00: Toggle
    • 01: Momentary
  2. CC number
  3. On value
  4. Off value
  5. Channel
 f7  SysEx end byte

Note that I’ve neglected exploring some of the options provided by the GUI, such as assigning the buttons to Program Change events. I also have no idea what the “Fixed” field does for the drum pads, as the word “Fixed” is the only description offered by the GUI. I suspect it enables fixed velocity, but I haven’t bothered to check.

Next Steps

Now that I’ve decoded the SysEx protocol, I plan to make a tool to enable editing the controller’s configuration under Linux. I may go as far as writing a controller script to enable modifying it from within Bitwig. Check for updates here to see what’s to come.

Generating spectrograms the hard way with numpy.

A spectrogram is a convenient visualization of the frequencies present in an audio clip. Generating one involves obtaining the frequency components of each window of the audio via a Discrete Fourier Transform (DFT) of its waveform. While tools are available to both generate spectrograms and compute DFTs, I thought it would be fun to implement both myself in my language of choice, Python.

In the following, I will discuss computing a DFT (the hard way), processing a WAV file, and rendering a spectrogram (all in Python). If you’re impatient and just want to see the code, you can find it on GitHub.

Continue reading Generating spectrograms the hard way with numpy.

Integrating GitLab and Google Calendar.

Zeall, like many other software startups, uses GitLab for version control and issue management. We also use the ever-popular Google Calendar to handle meetings, reminders, and deadlines. For several months, we’ve been looking for a way to automatically push GitLab issue deadlines into Google Calendar, and until now it seemed impossible. Only after a recent migration from our own private mailserver to G Suite did we find a solution — or rather, figure out how to feasibly build one.

Continue reading Integrating GitLab and Google Calendar.

The fruits of some recent Arduino mischief.

I recently consulted on a project involving embedded devices. Like most early-stage embedded endeavors, it currently consists of an Arduino and a bunch of off-the-shelf peripherals. During the project, I developed two small libraries (unrelated to the main focus of the project) which I’m open-sourcing today.

Continue reading The fruits of some recent Arduino mischief.

A simple recommender system in Python.

Inspired by this post I found about clustering analysis over a dataset of Scotch tasting notes, I decided to try my hand at writing a recommender that works with the same dataset. The dataset conveniently rates each whisky on a scale from 0 to 4 in each of 12 flavor categories.

Continue reading A simple recommender system in Python.

An easy way to visualize git activity

Today, I wrote gitply — a fairly simple Python script for visualizing the weekly activity of each contributor to a git repository.

It started out as a run-once script to get some statistics for one of my projects, but I ended up improving it incrementally until it turned into something friendly enough for other people to use.

Continue reading An easy way to visualize git activity

Adventures in image glitching

Databending is a type of glitch art wherein image files are intentionally corrupted in order to produce an aesthetic effect. Traditionally, these effects are produced by manually manipulating the compressed data in an image file. As a result, this is a trial-and-error process; often, edits will result in the file being completely corrupted and unopenable.

Someone recently asked me whether I knew why databending different types of image files produces different effects — and particularly, why PNG glitches are the most interesting. I didn’t know the answer, but the question inspired me to do a little research (mostly reading the Wikipedia articles about the compression techniques used in different image formats). I discovered that most compression techniques are not all that different. Most of them just employ some kind of run-length encoding or dictionary encoding, and then a prefix-free coding step. The subtle differences between the compression algorithms could not explain the wildly different effects we observed (except for in JPEGs, perhaps, since the compression is done in the frequency domain). However, PNG used a pre-filtering step which made it stand out.

Continue reading Adventures in image glitching

A fun experiment with Twilio

I first heard about Twilio a long, long time ago. As Google Voice faded out of relevance, it took the lead in the mobile-communication-as-a-service market. However, I had never had the chance (or inclination) to play around with its API until today.

About 12 hours after we landed back in the US from our holiday in Mexico, Lynsey departed once again — this time to the Plant and Animal Genome conference (PAG) in San Diego. She asked me to supply her with pictures of our cats for the duration of her trip. I told her I would send her a cat pic every hour, on the hour.

I didn’t realize what I had gotten myself into until I had already deposited $20 into a new Twilio account and spent 2 hours coding away… Though my goal was just to send some photos of cats, I had developed a pretty general application that lets you build a queue of MMSes to be disseminated at a constant rate.

Continue reading A fun experiment with Twilio

Time clocking at the command line

I often feel inclined to start new projects to avoid working on old ones. In a particularly ironic display of procrastination, I have written a productivity-oriented application in order to avoid actual productivity. The app is called InSTiL, and its goal is to make it easy to log how much time you spend working on various projects. The source is available on Github, and the Readme provides a succinct overview of InSTiL’s functionality.

https://github.com/le1ca/instil

A C++ encapsulation of the Linux inotify API

The inotify API allows you to monitor a file or directory for various events such as file creation, modification, and deletion. It is part of the Linux kernel and the glibc userspace library, however its C API can be cumbersome to use in a C++ application. A C++ binding of inotify does exist, but it still requires the application developer to write an unsightly wait-and-handle loop. My goal for this project was to create an asynchronous event-driven API through which filesystem events can be processed.

Continue reading A C++ encapsulation of the Linux inotify API