Category: Sound Design

Ars Electronica

Overall, I liked the Ars Electronica visit. The building itself is nice, the facilities are well designed, and Linz is already a beautiful city. I can imagine that during the Ars Electronica Festival it becomes a much more exciting place with more people, exhibitions, and events.

The project that caught my attention the most was Solar Synthesizer 0.4. I liked the idea of making electronic music with solar energy and controlling it just by changing the light with your hand. It is a simple concept, but I think it works well because it combines music, interaction, and sustainability. Even though I found the technology behind it a bit outdated compared to what is possible today, I still think the idea is interesting and creative.

My only criticism of the exhibition in general is that some of the technology and especially the AI-related installations felt a bit outdated. AI is developing so fast that I expected to see newer and more advanced examples. I think there are now other places in the world where more experimental and up-to-date technologies can be experienced.

Still, as a student studying sound and interaction design, I think it was definitely worth visiting. It is a good place to get inspiration and see different approaches to interactive media, even if not every installation feels cutting-edge anymore.

Paris and IRCAM Excursion

Paris was beautiful. Since it was my first time seeing the city, I do not really have any complaints about the city itself. The museums we visited and the areas we explored were useful, carefully maintained and special places. It was also good to experience the atmosphere of Paris outside of only reading or hearing about it.

Naturally, the trip was very expensive. My criticism is more about the support around the excursion. I think the school did not provide enough support, and some of the support that was promised did not arrive in a timely way. For example, the IRCAM conference ticket was not a surprising or unexpected cost. It was a expected fee from the beginning, so I think the school could have at least covered that part. They did not, and that was frustrating. Still, in the end, I had the opportunity to see Paris, and I am happy about that.

Apart from the financial side, I liked visiting IRCAM. It is clearly a very established institution, and it has pioneered many things in the field of sound, music technology and computer music. It is also valuable that they hold this conference every year and bring people together around these topics.

At the same time, I also felt a little disappointed. As many of my friends noticed as well, IRCAM felt a bit outdated and overly commercialised in some parts. Maybe this is normal for a large institution with a long history, but I expected some of the work to feel more experimental and more forward-looking.

Some of the people I met were helpful, and some were not, but that is not the most important point for me. What bothered me more was that some presentations felt too outdated for an institution that sees itself as a frontrunner in audio technologies. For example, there was a presentation about audio in gambling. In short, the presenter sonified gambling data in Pure Data. Of course, the patch itself was fine, and I do not want to disrespect the work. But as a topic, it felt very simple and outdated for a place like IRCAM.

This is only my personal opinion, but I expected a stronger level of experimentation and technological relevance. Still, the excursion was useful overall. Seeing Paris, visiting IRCAM and observing both the strong and weak sides of such an important institution helped me understand the field more realistically.

10. Current State and Personal Reflection

At this point, the project has reached a working prototype stage. It is not finished as a full thesis yet, but it is no longer only an idea or a sketch. There is now a real system that can be opened, configured and listened to. The listener can move through a virtual source layout, the system calculates direct-sound behaviour, and REAPER applies the values through the audio chain.

Looking back, the most important progress was not only technical. Of course, I learned a lot about Python, OSC, REAPER routing, mcfx, IEM plug-ins and binaural rendering. But maybe the bigger development was learning how to reduce the project to something I can actually defend. At the beginning, I wanted to work with physical sound superposition in a room. That idea is still important to me, but during the semester I understood that I first needed a controlled reference system.

The current prototype is therefore not trying to be everything. It does not model the full room. It does not include reflections, reverberation, diffraction, occlusion or source directivity. Instead, it focuses on the direct path between each source and the listener. This limitation is not only a weakness; it is also what makes the system understandable. I can explain what is calculated, what is sent to REAPER, and what is heard over headphones.

One useful lesson was that wording matters. If I call the system a room simulation, I create expectations that the prototype cannot fulfil. If I call it a direct-sound auralisation or a controlled reference model, the project becomes more honest. This was something I had to learn through feedback. Some parts of the system were already working, but my explanation was sometimes too broad or too confident. The feedback helped me clean that up.

The project also changed how I think about tools. I started with Pure Data and Max/MSP, but the final direction needed a custom interface and a more flexible calculation layer. PyGame and Python gave me that. REAPER then became the audio engine where the calculated values could be tested in a real plug-in chain. This combination is not always elegant, and sometimes it creates annoying technical problems, but it also gave me a system I understand from the inside.

The next step is to compare this controlled prototype with the physical room. That is where the original idea returns. The CUBE can show what the real room adds: reflections, room response, loudspeaker behaviour, interference and perception. I think this comparison is now the most interesting direction for the thesis. The prototype gives me the clean version; the physical room gives me the complicated version.

For me, the value of this semester is that the project became concrete. I now have a technical baseline, a clearer research direction and a better understanding of the limits. The system is not perfect, but it works, and more importantly, I can explain why it works the way it does. That feels like a good point to continue from next semester.

9. Next Semester: Bringing the Prototype to the Physical Room

The next step is to bring the project back to the physical room. The current prototype gives an idealised direct-sound condition. The CUBE will add the things that are deliberately missing: reflections, reverberation, room response, loudspeaker behaviour, source directivity, modal effects and real listener perception.

The goal is not to prove that the auralisation is “better” or that the real room is “worse”. That would be the wrong question. The more interesting question is: what changes when the same source layout moves from a controlled headphone-based auralisation into a physical multi-loudspeaker environment?

To make this comparison meaningful, the next semester needs a clearer method. One possible approach is to create selected source layouts in the GUI, then recreate equivalent layouts in the CUBE. The same or similar sine-tone frequency sets can be used, such as 440 Hz, 444 Hz, 448 Hz and related intervals. This keeps the connection to the original superposition idea.

For measurement, I do not want to claim a complete protocol yet. But the basic plan would be to use fixed source and listener positions. Level could be measured with a measurement microphone at selected listener positions and compared with the calculated direct-sound attenuation. Delay or time of flight could be checked using short impulse-like test signals and looking at the first arrival. The later reflections would then show what the room adds beyond the direct path.

Perception will also matter. The system can predict level and delay, but the listener may experience the physical room differently because of reflections, beating, instability or localisation changes. This is where a listening-test method could become important. It should focus on movement, localisation, beating and perceived spatial stability.

The game/interface side project may or may not connect to the thesis later. During the semester, I became interested in the PyGame controller and even made a Doom-like game, later moving it to Godot. That is not the main thesis right now, but it influenced how I think about movement and interaction. Maybe there will be a connection later; maybe not. For now, the safest thesis direction is the comparison between controlled direct-sound auralisation and real-room behaviour.

The project has reached a working baseline. The next step is to test what this baseline can reveal when it meets the physical space again.

2. Learning the Auralisation Idea Through Drawings and Discussion

One of the important turning points was a meeting where my professor explained the auralisation approach in a very practical way. He drew things on paper: source positions, listener position, signal flow, and how the values should move from a visual interface into audio processing. That was educational for me because it turned the project from a vague spatial-audio idea into a system I could actually build.

Before that, I understood the general concept, but not yet the full chain. I knew that I wanted sound to react to movement. I knew that source distance should affect level. I knew that direction should matter. But the system was still blurry. The drawings helped me understand that the project needed clear stages: an interface, a calculation part, OSC communication, REAPER, spatial audio plug-ins, and finally binaural playback.

This also helped me understand why the word “simulation” is dangerous. A full simulation would mean modelling the complete acoustic behaviour of a room. That includes reflections, reverberation, diffraction, occlusion, source directivity and probably much more. My system does not do that. It calculates the direct path between a source and a listener. That is already useful, but it is not the whole room.

So I started to think of the project as a controlled direct-sound auralisation. This means the prototype gives an audible version of a simple physical relationship: where is the source, where is the listener, how far apart are they, what direction is the source, how much should the level decrease, and how late should the sound arrive?

That framing made the project less grand but much stronger. It also made the next semester clearer. If the prototype represents an ideal direct-sound condition, then the real room can later be understood as the thing that adds complexity. The CUBE will add reflections, room response, interference, loudspeaker behaviour and real perception.

In that sense, the drawings were not just technical notes. They changed how I thought about the thesis. Instead of trying to make a huge system that claims too much, I could build something smaller but testable. That was probably one of the most important lessons in the process.

It also changed my confidence. I had something concrete to implement. The project became a set of solvable problems instead of one giant abstract spatial-audio dream.

1. From Physical Sound Superposition to a New Direction

At the beginning, the project had a different shape in my mind. Last semester I was mainly interested in physical sound superposition: real loudspeakers in a room, sine tones playing at close frequencies, and the listener walking through zones where the sound changes. I liked the idea that the room itself could become a kind of instrument. The piece was not only about listening to tones, but about moving through them and discovering how interference, beating and phase relationships appear in space.

That first idea was more installation-based. I imagined loudspeakers placed in a room and the listener moving physically between them. The focus was on the direct experience of sound in space, not on a screen or an interface. This still feels important to me. Even now, after the project has changed, the original interest is still there: movement, sine tones, spatial relationships, and the strange physical feeling that happens when simple sounds interact.

But as the semester continued, the project moved away from being only a physical loudspeaker installation. In meetings with my professor, it became clear that I needed a more controlled research step before going into the real room. If I directly built a physical installation, many things would happen at once: reflections, room modes, loudspeaker differences, directivity, occlusion, and the listener’s own perception. That can be interesting artistically, but it is difficult to study clearly.

This is where the idea of auralisation entered the project. Instead of trying to simulate the full room, the new direction became: build a controlled direct-sound reference first. In this version, I can define source positions, listener position, distance, direction, level and delay. Then later I can compare that controlled version with the physical room.

So the project did not completely abandon the first idea. It changed its method. The original project was about sound interaction in a real space. The current project builds a headphone-based interactive prototype that can later be tested against a real multi-loudspeaker room. I think this change made the project more realistic as a thesis. It still comes from the same curiosity, but it now has a clearer technical and research structure.

The main question also became more careful. I am no longer asking whether I can completely simulate a room. That would be too large and too easy to overclaim. Instead, the current question is closer to this: how can an interactive binaural auralisation help compare ideal direct-sound behaviour with a real multi-loudspeaker room?

That change in wording matters. It gives the project a better foundation. I can say what the system does, but also what it does not do yet.

Overall I must say, it was quite a journey. Navigating my way through the various errors and setbacks I faced was truly soul crushing at times. I often didn’t know how to proceed and felt completely frozen and helpless. However, I always received assistance from my mentor, Mr. Sontacchi, who motivated me to push through till the end. He has guided me through many obstacles I faced and helped me re-establish interest in the project when I thought I couldn’t make it happen. I would like to personally thank him for all the effort he has put in and the support he’s given me throughout this semester.

One thing I wish I had was more time to focus specifically on this project. There were various scenarios where I was caught between multiple deadlines, which restricted me from focusing on what I wanted to achieve. It was frustrating as I wanted to progress and move forward with the work but was held back due to the workload from the other courses. Balancing the time for this research project alongside the other project felt difficult. I felt this project could have been even more expansive if I had more time to contribute to it. 

One of the things that I had to change was the model of the waveform for the visualizer. I initially intended to work on a 3D waveform, with more particle clouds and points, but it turned out to be more complicated than I had initially expected. Through my mentor’s guidance, I built the existing model, which varies in pitch based on its position on the y-axis. I really like the current model, but would like to rework it and build an alternative for the final installation. 

Additionally, I wish to integrate the pitch to colour mapping in some form. I am planning to map harmonics and timbre to the properties of the newly designed visualizer. I also need to implement a surround sound system for the audio response to recreate an immersive environment, as well as offer additional control parameters such as playback loudness, directional panning and wet/dry mix of the output. There is a lot of testing that needs to be done to ensure we don’t suffer from latency issues.

In the end I am happy with the output and super glad to have a functioning prototype to present. That being said, I still believe there is a lot of work to be done in the upcoming semester for me to completely realize the installation that I had envisioned. I am looking forward to continuing work on this research project.

My prototype for this semester is finally complete. After much trial and error, I was able to build a visualizer that functions as per the user’s input and is represented in a model that is easily understandable to the user. The waveform morphs and reacts based on the user input and doesn’t break or alter its core geometry in the process, finally maintaining a smooth and fluid visual experience. There are still a few additional tweaks that can be done, but it’s somewhat functional at the moment, which means that I have met my primary goals for this semester.

Some additions include the audio feedback response, which acts as a monitor based on chords to determine the pitch accuracy of the user. This was done using the quality parameter in the sigmund object, a very useful tool for pitch tracking. The signal runs through a relational operator (>) and when the signal fulfills the condition (greater than 90%), the patch plays back a chord for the user from an oscillator based on their pitch quality. If the user is perfectly in pitch, it will play a pleasant major chord, while a slightly off-pitch input would trigger a more dissonant minor chord, providing immediate auditory feedback without the user needing to look at a screen.

Another addition is the waveform, which now features a visual activation response where the visualiser remains inactive or dark when there is little to no input signal. The colour in the waveform reappears when the user is speaking, and this is a vital visual feature to indicate that the signal is being successfully received from the user. By fading to dark during silence, the interface reduces visual fatigue and makes the interactive moments feel much more dynamic. This addition was done using the data from the quality argument in the sigmund object, which was mapped to a Logic CHOP in TouchDesigner to trigger the visual state changes seamlessly.

I also considered adding a few advanced audio analysis tools from a Python library to give users more speech information. What I had imagined was an algorithm which collects a live data feed and predicts the different groups of consonants present in the speech (fricatives, affricates, plosives, etc). You would see numbers fluctuating on screen showing the variations in frequency as well as the tonal quality.  Unfortunately, I won’t have enough time to implement this into the prototype, and it would be considered as a feature to be added in the future.

In the final blog, I will share my overall experience building the prototype, reflect on the challenges, and give insights on what to look forward to in the upcoming semester.

Last month, the sound designers and exhibition designers of FH Joanneum jointly participated in an excursion to the Ars Electronica Center in Linz. The center is a museum that focuses on new media art and is most renowned for its annual festival, the Ars Electronica Festival, which critics have praised as a pivotal exhibition in the digital arts. It is widely considered one of the best known creative arts festivals in Austria. Over the years, the people behind the center have been praised for their unique integration of technology into the creative field. They also have a dedicated inhouse research facility called the Futurelab, which focuses on modern technological advancements, particularly artificial intelligence. 

There were a lot of engaging exhibits. Some memorable ones included the fascinating “tardigrade” under a microscope, as it made its way through its tiny surroundings. These remarkable creatures are incredibly resilient, capable of withstanding extreme temperatures and are known to be some of the only organisms able to survive in outer space. Another interesting part of the excursion was the Deep Space 8K experience, which drew us into a world of 3D visuals. We heard various soundscapes, ranging from the planets of our solar system to our oceans, and even the unique art of yodelling.

The exhibit I would like to speak about in detail is the simple sequencer, which was located in the Kids Research Lab on the first floor of the building. I really enjoyed the sequencer and felt like a kid again while playing around with it for some time. It was a really nice setup, especially for children who are curious about music and sound in general. It was a simple concept, but one that was very much fun to use due to the various patterns you could recreate.

The sequencer was off by default, so you had to place blocks (or tiles) representing a particular instrument onto the vacant slots to produce sounds. On the left hand side of the instrument block was the fill pattern, consisting of 8 distinct beats that the user could customize to recreate a unique rhythm. There was an LED strip on the top edge of the sequencer, which indicated the beat being played at the time. The beat blocks came in different colours and sizes. Some were taller than the others and had unique shapes on the top facing size. However, I was not able to decipher exactly what these could have stood for. This may have possibly been a measure used for quarter or half notes, but I am not entirely sure. The instrument blocks included an acoustic guitar, a piano, an electric bass and even a saxophone.

The sequencer could only accommodate 4 instruments at a time as there were only 4 slots available. Apart from this, there was also a small blue box on the right side of the sequencer with buttons featuring images of snares, cymbals and kick drums. I do not recall if these sounds played in sync with the sequencer but pressing them did produce the sounds of the instruments I just described.

Overall, I really enjoyed the visit and gained valuable insights from the unique exhibits. From the way they were presented to the overall setup of the installations, it was a good trip to explore and brainstorm ideas for our own installations at Klanglicht later this year.