The first step to build my working prototype was to start with the Pure Data patch. This is an integral part of the project as all the live data is being collected and parsed through these connections. I used an alternate version of Pure data called plugdata. I found it more convenient to work with as it has a much more modern UI, helping users (especially me) navigate the software easily.
So, my intention behind this patch was to build a system to collect live audio signals, retrieve information about pitch and loudness of these signals, and store them in lists which can be sent to OSC for TouchDesigner to collect and interpret. The audio signal is captured using an [adc~] object which is then amplified using a [*~] object which is connected to a horizontal slider to ensure input gain can be controlled by the user. This is one of our most important control parameters. The patch then analyzes the audio stream and splits it into the two properties that I mentioned earlier, pitch and loudness.
In order to obtain loudness information, we had to use the RMS value which can be computed using the [env~] object. I wanted to use decibels for the project, as it is the most commonly used measurement for playback systems. In order to achieve this, I used the [rmstodb] object. To test if the signal was functioning as expected, a VU meter was connected to monitor the signal. This was then connected to an oscsend for the data to be transmitted.
As for the pitch, it was slightly more complex. I had to use the [sigmund~] object with the arguments ‘-pitch’ and ‘-peaks’ to obtain the values. I was not familiar with this object and it took me some time to understand the working behind it. It continuously analyzes the waveform to track fundamental frequency. The frequency value is then passed through an [ftom] object which converts frequency into MIDI objects. This helps to work with musical notes that are more familiar. It is also much easier to map to visual parameters. These MIDI values are also sent through OSC for TouchDesigner to collect and process.
My next step is to test the OSC data and ensure it is being received correctly by TouchDesigner. Should be simpler considering the data being transferred locally. Localhost should suffice for this and netsend might also do the trick. My findings will be revealed in the next blog. Stay tuned!
