Iola Walker: A Mobile Footfall Detection System for Music Composition

TL;DR

This work presents iola walker, a mobile footfall detection system intended to enable gait-synced music composition and playback. It deploys a Conv-BI-LSTM architecture with two 1D convolutions and a bidirectional LSTM, trained offline on 200 Hz IMU data and exported for on-device inference via TorchScript. The pipeline covers data acquisition, model training, and a streaming inference loop with smoothing and tunable parameters to achieve low-latency interaction. Overall, the study demonstrates a practical framework for integrating real-time gait signals into musical experiences and invites open-source experimentation and future refinements.

Abstract

This outing is part of a larger music technology research project. The objective is to find a way to enhance music using hardware and software. This is the documentation for the Whimsical first part of the research project: it's an android app that detects a wearer's footfalls by running live inference on an LSTM. The system works by getting data from an Mbient Labs IMU to a mobile app over bluetooth. After you move the .csv file to a computer with a GPU, you can use the python code to train an LSTM on that data. You then export the LSTM to the android app and can begin detecting footfalls. Feel free to download and experiment with the code. It's meant to be read and improved upon by you and your LLM codewriter of choice! https://github.com/willbjames/iolawalker