After exploring the theoretical and physical principles behind the footstep phenomenon, it was time to bring the concept into the digital realm using signal processing and computer vision. The first experiments of this project took place in the plugdata environment and dealt with manipulating the playback speed and amplitude envelope curves of pre-recorded footstep samples.
First sample based test patch
Slowing down and speeding up a sample during playback was not a practical long-term approach, but it served as a quick first sketch to develop the concept upon. At this stage, it became clear that I needed to construct a control interface capable of taking very detailed control over the timing of the heel, roll, and toe stages. I also attempted a spectral morphing process to combine a footstep sample with a constant texture sound, but it lacked the organic feel I was aiming for.
Another early trial involved receiving Open Sound Control messages from a computer vision Python script for the very first time. I mapped the outer fingertips of the user’s pinky fingers on an X and Y axis across the webcam frame, allowing hand movements to manipulate the frequency and amplitude of a simple sine wave.
First plugdata patch to receive OSC messages from Computer Vision
A major milestone in this process was translating these theoretical concepts into the project using Andy Farnell’s procedural model for footsteps. His method divides the architecture into an upper control mechanism and a two-stage synthesis section. In the first stage, a master phase signal acts as the main driver, splitting into independent foot phase controls to activate the correct locomotion muscles. If the simulated walking speed exceeds a certain amount, the overlap of the phases diminishes, automatically transitioning the system from a walk into a run. In the second stage, this control signal gets translated into a physical ground response force, which is then fed into dedicated sound-generating patches designed to mathematically synthesize specific textures like snow, dirt, or gravel.
Overview footstep pd patch. Farnell 2010, 551
Farnell, Andy. Designing Sound. Cambridge: The MIT Press, 2010.
