While computer vision serves as a highly viable mechanism for translating kinetic energy into digital control data, relying on hardware sensors might be a clever decision to gain more accuracy and reliability at the slight expense of convenience. Since this project revolves around footsteps, it makes perfect sense to integrate data-receiving tools into a mobile floor surface, effectively creating a modernized, digital Foley pit.
This proposed device would consist of multiple layers, measuring approximately one square meter in size. The top layer would feature a hard acoustic surface, such as a premium hardwood floor, while the bottom would be heavily decoupled from the surrounding environment using a sophisticated rubber and foam dampening system. This acoustic isolation is crucial to prevent the system from picking up ambient room interference that could trigger the digital signal chain unintentionally. Sandwiched between these two layers would be multiple piezoelectric contact microphones designed to instantly pick up the physical impacts and route them straight into the plugdata environment.
Personal sketch of the piezoelectric floor sensor device
By implementing this hardware approach, it could trigger the exact footstep timing and velocity by performing actual steps on the surface. This physical connection would inherently lead to a much more natural human gait reproduction. At the same time, both of the performer’s hands would remain entirely free to execute subtle real-time changes to the acoustic parameters via computer vision, seamlessly morphing surface textures or shoe materials while completely eliminating the slight latency associated with camera-based motion tracking.
