Last month, I had the opportunity to attend the IRCAM Forum in Paris with my classmates. It was very enjoyable and provided a pathway to explore a variety of projects in the innovative fields of sound technology and research. The workshops were generally well presented and featured discussions that helped attendees better understand the concepts and workflow. Some of these projects really stood out to me, and I would like to share them in detail throughout this set of blog posts.
“Liberated frequencies” is a Bio-interactive Audiovisual performance by Keigo Yoshida which combines neuroscientific research with machine learning to generate sounds in real-time. During this performance, an AI is trained to learn favorable patterns and generate sounds from signals triggered by the user’s EEG sensors. In other words, the brainwaves (theta waves in particular) of the user are measured and sonified to form abstract compositions from preferred sound textures.
The team behind the installation collected a library of glitch, experimental, voice and noise sounds, which the subject rated based on the pleasure they evoked. During the performance, the AI learns in real-time from the most preferred sounds. However, instead of simply creating a comforting environment, the AI utilizes this data to intentionally disrupt the experience. Once high auditory pleasure is detected, the system alters the pitch, tempo, and rhythm, pulling the listener out of their comfort zone to challenge human-centric aesthetics and create an unpredictable, evolving soundscape.
The RAVE (Realtime Audio Variational autoEncoder) framework was used to process and generate the sounds, while the visual representations were made and mapped through TouchDesigner. The translation of data to an audiovisual form in real-time is one of the primary aspects of my research for my personal project. Therefore, I remain open to explore more in the world of real-time data audification and visualisation. This brings me to the next workshop, which is also based on neuroscientific studies.
Brainwave-Modulated Generative Music
This live demo presentation was done by Zap Bain, musician, sound engineer researching on auditory neurofeedback and spatial audio. This project also dwells on real-time EEG brainwave control of spatial audio systems. Despite not being familiar with neuroscience, I found the research very fascinating and it made me curious about neurofeedback responses in general.
The artist used embodied cognition techniques including movement, neural asymmetry, vestibular system engagement and brainwave frequency ratios for this research. The presentation showcases ambisonic work developed with SPAES using Spat~ and boids algorithms.
I was able to link some of the methodology behind this presentation to my own research project. I was exploring ways to utilize sounds in a surround sound/ambisonic setup just using vocal feedback and this session gave me some hints on how I could integrate different parameters to control the sound signals and how it could be linked with the visual element to make it more understandable.
