Month: July 2026

In May, we took a day trip to Linz, a UNESCO City of Media Arts, to check out the ARS Electronica Center, an institute and creative ecosystem combining the fields of art, science, and technology. In this blog entry, I will cover some projects that caught my attention.

Biology and Art

One of my highlights was seeing the research and various approaches stemming from biology.

For example, Interwoven, an artwork by visual artist Diana Scherer. By studying plant root growth, she developed a biotechnique in the form of various templates that the root system adapts to. The result shown was an intricate, textile-like pattern. This is why I was drawn to this research, as the resulting patterns were a mix of human intervention and biological adaptivity, creating various interesting textures and shapes with naturally occurring randomness rather than digital computation.

One more textile design practice I was drawn to was Growing Patterns, Living Pigments by Julia Moser, which explores dyeing processes via pigment-producing bacteria in combination with various techniques. The bacteria used, which naturally produces a bluish-purple pigment called violacein, reveals organic patterns and hues, as well as showcase a potentially sustainable approach to textile coloration.

Both of these works are based on co-creation with non-human biology. Organoids by Jürgen Knoblich and Madeline Lancaster, in a way flips the script by using human stem cells as the starting point. Created for scientific research rather than artistic production, the project focuses on organoids, masses of cells grown from stem cells with brain-like structures similar to the embryonic brain. The project gives new insights into how the human brain is formed, which can help researchers understand the development of brain disorders.

~21,600 by Dorotea Dolinšek looks at the human through the lens of an organism defined by an intrinsic, vital dependence on air. The resulting version shown at ARS was a prototype of the work, made by blowing coal dust onto paper over the course of 9 hours, using her breath as the central gesture. This work interested me due to its results: a collection of coal dust dots, none exactly the same, creating a pattern that could be interpreted as an algorithm and potentially used for sonification and algorithmic composition. In the end, she made it into a video performance spanning 24 hours, featuring a mask with a microphone as the input and an airbrush attached to the nose of the mask as the output.

AI and Art

During our visit, we got a tour of the Piano Room, equipped with a Yamaha DC1, a grand piano capable of self-playing, attached to an AI-trained algorithm for visualization developed at Ars Electronica Futurelab. For a practical demonstration, I got to play an improvisation. It was intriguing to see the resulting visuals, as the algorithm was trained mostly on classical music. While playing a repeating phrase, it visually adapted and formed a sort of memory of the visual pattern, referencing it for a short moment afterward.

While in no way new, it was still interesting to see and interact with installations using GANs (generative adversarial networks), a generative AI framework comprised of two neural networks that compete against one another to create synthetic data.

Pix2Pix: GANgadse lets users draw sketches, which are turned into cat pictures. This was done by feeding the algorithm cat images and training it to recognize typical features, like pointy ears and cat-shaped eyes. In practice, it reads your drawn shapes and approximates a guess. This was interesting to play around with, as drawing nonsensical sketches produced abstract and slightly unsettling flesh-like objects. We also tried drawing an elephant, which it turned into a fuzzy creature, and a cat, which worked as intended.

ShadowGAN uses a camera to capture the outlines of visitors and, using the algorithm, reinterprets those silhouettes as landscapes. In this case, the AI was trained with images of mountains and their landscapes, with the aim of interpreting everything it sees through this lens. The result was an interactive installation comprised of various abstract landscapes.

Both of these projects create digital forms that adapt to incoming input, almost like digital organisms.

3D 8K Experience

Lastly, I’d like to mention Deep Space, a 3D experience space developed by the Ars Electronica Futurelab. The room consists of eight 4K projectors, able to produce 8K when paired in twos, projected onto the wall and floor to create one continuous space. This is combined with 3D shutter glasses for visitors and a laser tracking system, though in our case, the interactive tracking capability wasn’t put to full use, since we experienced the program seated rather than moving through the space ourselves. The space also features a modular multichannel audio system.

Our guided tour through the experience consisted of nautical and space themes, with the most memorable moments being a 3D projection of a whale, followed by an artistic piece consisting of 3D-projected lines engulfing the audience that act as a translation and visualization of a musical composition.

Sources:

https://ars.electronica.art/news/en/
https://ars.electronica.art/futurelab/en/projects-deep-space-evolution/
https://dianascherer.nl/
https://www.livingpigments.com/
https://ars.electronica.art/center/en/organoids/
https://www.doroteadolinsek.com/projects/21-600-2
https://ars.electronica.art/center/en/piano-room-2/
https://en.wikipedia.org/wiki/Generative_adversarial_network
https://ars.electronica.art/futurelab/en/projects-understanding-ai/
https://ars.electronica.art/center/en/pix2pix-gangadse/
https://ars.electronica.art/solutions/en/deepspace/
https://ars.electronica.art/futurelab/en/projects-deep-space-evolution/

This blog is the second of a two-part retrospective on the excursion that we, the first-year sound design students, took to Paris. The topic at hand is a few activities outside the scope of IRCAM that I would like to highlight for being memorable. If you are interested in some of the happenings at IRCAM, you can check out the previous blog.

Audio Guide Tour Through the Hôtel de la Marine

Firstly, I would like to mention the location-based audio guide at the Hôtel de la Marine, an eighteenth-century historic monument and museum overlooking Place de la Concorde in the 8th arrondissement of Paris. It uses binaural sound technology with over-ear headphones that trigger different parts of the story depending on your location within the building. The over-ear design of the headphones worked surprisingly well, with audio quality that was noticeably higher than the usual museum experience.

The audio guide covers a significant portion of the building’s history, spanning more than two centuries. Covering all of this in just two hours occasionally made the story confusing due to the amount of information and the narration jumping between various historical moments. That said, the voice acting and audio dramas were immersive and well-produced.

One moment that particularly stood out to me was when the audio guide prompted me to look through a small opening in the wooden shutters, explaining how it had been used during the German occupation of Paris. As the story unfolded, it shifted its focus to the building across the square, mentioning a Nazi flag that once flew there before moving to the moment when the French Tricolour triumphantly took its place again. Seeing the flag flying in the same position on the building across the street was an example of how the immersion and interaction of the experience can be enriched to an even greater degree.

Philharmonie de Paris Musée de la musique

Another visit I enjoyed was to the Musée de la musique at the Philharmonie de Paris, located in the 19th arrondissement. The museum houses a large collection of instruments covering several centuries and cultures. Before entering the exhibition space, we were each given an audio guide. The highlight for me was the wide range of instruments, especially those that I could hear as audio samples through the guide. We also had the opportunity to hear a live performance featuring the erhu, sometimes referred to in the Western world as the Chinese violin.

The museum’s extensive collection felt somewhat overwhelming. We spent what felt like three to four hours there, and I still was not able to properly get through all the instruments I was interested in. I was surprised by the number of instruments catalogued in the audio guide app. I especially enjoyed learning about instruments that featured both a video demonstration and a detailed description. It is a shame that not all instruments are covered to this degree, with the level of documentation varying from highly detailed to almost nonexistent.

It was also nice to see the collection of electronic instruments, although it was somewhat sparse compared to the other periods represented in the museum. The Hammond Model A organ, Minimoog Model D, and UPIC, an interesting visual music computer invented by the avant-garde composer Iannis Xenakis, stood out the most. I would also like to mention the collection of instruments featuring fantastical animals as decorative elements, such as a nineteenth-century French saxhorn with a decorative head whose tongue flails around when it is played.

We were also fortunate enough to view the reconstructed studio of the musique concrète composer Pierre Henry, which was actually closed to the public that day due to renovation. It featured several interactive modules aimed at understanding Pierre Henry’s work in a more hands-on way. My favourite was the tape machine module, which produced solid results from a simple setup.

Sound Installation at the Pleyel Footbridge

Lastly, located in Saint-Denis, a commune in the northern suburbs of Paris, we visited the Pleyel Bridge, a crossing tied to the new Saint-Denis Pleyel transit hub. Here, we viewed Promenade Sonore: Vent, Soleil, Pluie by the Swiss sound artist and landscape architect Nadine Schütz. Installed on the bridge as a collaboration with the architect and engineer Marc Mimram, as well as in part with IRCAM, the work transforms the supporting structure into sculptural sound-generating instruments, each corresponding to a meteorological element. We were able to experience two of the sonic sculptures, Soleil (sun) and Vent (wind).

Soleil uses luminosity, temperature, and the time of year to evolve its soundtracks and their effects. It consists of a steel sheet supporting structure housing 24 transducers used to resonate the surface. The sculpture invites pedestrians to press their bodies against it in order to feel and hear the vibrations of the composition. This is an element I particularly enjoyed, as it adds an interactive aspect with varying and localized sonic results.

Vent, located further along the bridge, turns wind vibrations into a musical layer. Sixteen tensioned strings are stretched across the bridge arches and connected to piezoelectric microphones, which capture the vibrations caused by the wind. This installation stood out to me due to its interaction with the trains passing below, resulting in a mixture of environmental and generated sonic content.

From my perspective, the soundscapes of the bridge do not forcefully impose themselves on the passerby but rather aim to enhance the atmosphere of the space. Viewed through this lens, the installation reflects a degree of human-centred design. Human-oriented choices also appear in the architecture of the adjacent Saint-Denis Pleyel Station. Designed by Japanese architect Kengo Kuma, the station incorporates fireproofed wooden beams throughout its structure, adding visual warmth while also functioning as acoustic treatment. Rather than allowing the harsh noise and reverberation commonly associated with concrete transit hubs to dominate the environment, the wooden elements help soften and regulate the interior soundscape.

While the station seeks to reduce and control some unwanted acoustic effects, the installation by Nadine Schütz takes the opposite approach by introducing new sonic elements into the environment. Together, they demonstrate how human-oriented acoustic design can be achieved either via reduction or via extension.

Sources:

This blog is the first of a two-part retrospective on the excursion that we, the first-year sound design students, took to Paris. The trip took place in March and centred around the IRCAM Forum Workshops, a week of presentations, workshops, and demonstrations hosted by IRCAM, a French institute for research in music and sound. The programme ran from Tuesday to Saturday across two venues. In this post, I will highlight two sessions that stood out to me from each location.

GRM Tools Atelier Workshop

Starting off with a workshop which took place at the IRCAM building (located alongside the Centre Pompidou on Place Igor Stravinsky), the presentation introduced GRM Tools Atelier, a modular real-time environment for sound processing and synthesis developed by INA GRM. The developers behind the project demonstrated it as a creative space where sound can be built, transformed, and explored through a network of connected modules. The main focus was on experimentation, encouraging users to discover new sounds by combining simple building blocks in somewhat unconventional ways.

For me, the highlight was seeing their comb filter both on its own and in combination with other modules. I was surprised by the variety of results that could be achieved with a simple sound synthesis concept and by the potential the plugin itself offered through additional controls such as harmonic manipulation, morphing, and interaction with other modules.

That being said, as a student, purchasing software like this is not always realistic, and in this case, I am not convinced owning the plugin itself is necessarily the most important takeaway. The principles demonstrated during the session can be applied using open-source platforms, like SuperCollider. Ultimately, the value I took from this session was less about the software itself and more about the ideas behind it. What initially appeared to be a presentation of a new tool was, from my perspective, a sound design lesson disguised as a software demonstration, and I am grateful for the experience, as it encouraged me to think about familiar processes in a broader creative context.

Demonstration of an Augmented Instrument

Also taking place at the IRCAM building, as part of Remix (an IRCAM showcase presenting the first results of several hybrid artistic research groups), the presentation by Katalin Koltai, a guitarist, researcher, and lecturer at the Royal Academy of Music in London, focused on the progress of her ongoing research into augmenting the classical guitar. In this context, “augmented” refers to a traditional acoustic instrument that has been enhanced with additional mechanical and/or electronic components to expand its sonic capabilities while maintaining the feel and playing techniques of the original instrument.

Their form of augmentation comes in the shape of the Open Frets Guitar, a prototype featuring a reconfigurable magnetic fretboard that allows the layout of the frets to be altered. This opens up possibilities for alternative tunings and microtonal performance, which stood out to me as I usually use non-standard tunings in my own guitar practice. Alongside this, the prototype featured the use of exciters, a type of electroacoustic transducer, used to make the strings and body of the guitar resonate.

The use of exciters also reminded me of a piece I made, 3.2.2024., for which I placed an exciter beneath the soundboard of a concert piano. The sound coming from the exciter was used to vibrate the body and strings of the piano, creating a natural filter whose strength depended on the amount of sustain pedal used. This practice is not new, but I personally enjoy seeing what others come up with and the different ways in which they enrich the sound. That was the general highlight of the presentation for me.

An Audiovisual Performance using Setar

Moving to the second location of the excursion, the transdisciplinary venue Centre des Arts in Enghien-les-Bains, a performance rather than a talk that stood out to me was Moving With Time by Seyed Ali Hosseini, a composer and researcher in electronic music; and Giuseppe Messineo, a composer and coder working with interactive audiovisual systems. The piece was an audiovisual real-time interaction performance exploring the evolving relationship between sound, image, and culture.

My highlight was definitely the beginning of the performance, which involved the use of the setar, a traditional Iranian lute, alongside vocal chops and visuals generated through TouchDesigner. Combined with the spatial audio setup, in which speakers were arranged in a circular formation, the performance created an atmosphere that was easy to become immersed in. As the piece progressed, so did the sound material and its context. As the name of the composition implies, it evolved over time, doing so by blending elements of electroacoustic music with a more dance-oriented approach.

Something I learned after reading more about the project was the use of concatenative synthesis, a type of synthesis that builds sounds by stitching together short samples from a larger sound database. Unlike granular synthesis, another sample-based technique that they used, concatenative synthesis allows recorded material to be analysed and reorganised based on specific characteristics rather than by manipulating small sound grains directly. The combination of these techniques allowed the original acoustic material to gradually transform into something much further removed from its source.

EEG Audio-Visual Performance on Pleasure

Lastly, I’d like to mention one more performance, which also took place at CDA, Liberated Frequencies by Keigo Yoshida, a Japanese artist and doctoral researcher whom I met during the forum week.

The performance was an audiovisual exploration of the relationship between artificial intelligence and human perception of sound. Alongside his assistant and narrator, Rinko Oka, they monitored brain activity using EEG sensors, a type of sensor used to record the electrical activity of the brain, and isolate brain waves related to pleasure. During the development process, Yoshida had previously tested his collaborator’s perception of different sounds, collecting responses to various sonic materials. These responses were then used alongside EEG during the performance, allowing the system to react to the collaborator’s auditory perception. The visuals were done in TouchDesigner.

I found the dialogue between the listener’s preferences and the system particularly interesting. Rather than moving towards comfort and familiarity, it deliberately generated a harsher soundscape, leaning on post-human-centred aesthetics in which human preference is no longer the only measure of value.

Sources:

This blog is a summary and review of Kevin Blackistone’s NIME paper, “Ultrasound Probe as Tool for Tangible Sound Performance Using Physically Sculpted Phantoms”. I wasn’t entirely sure what to expect, as my first association with the word “phantom” was more in line with some type of ghosting effect (a visual trailing artifact behind moving objects). What I was met with instead was a deep dive into medical imaging techniques, material exploration, and its sonification.

Core Concept: Medical Imaging Techniques

This paper documents a project centered around a standard handheld medical ultrasound probe. While I did get an ultrasound in the past myself, I didn’t know the technology behind it. After some research on the side, I have learned that this device is a type of transducer, transducers being devices that convert energy from one form to another (a category in which microphones and speakers also reside). It relies on the piezoelectric effect, which allows the probe’s crystals to vibrate and emit high-frequency sound waves, also known as ultrasound. As these sound waves travel or vibrate into the body, they reflect back to the device as echoes, which are then used to create a real-time visual interface. The NIME paper documents various movements when operating the probe and how they affect the resulting image.

While these devices are typically used on human tissue, medical training often relies on “phantoms,” which are opaque, dyed gelatin molds designed to mimic organic tissue, with the aim of helping students practice handling the probe and other sub-dermal (just beneath the skin) tools. Blackistone uses the ultrasonic medical probe in combination with DIY phantoms to create a physically expressive and sonic performance.

On DIY Phantom Design

The DIY phantoms used were composed of everyday food items such as tofu, grapes, orange peels, mushrooms, and even coffee grounds, with the aim of creating different membranes, materials, and surfaces, each of which reflects ultrasound differently, producing unique spectral characteristics once converted into sound via an additive synthesis approach. It is also a positive ethical choice to use organic matter, as it minimizes waste.

In my opinion, this approach of using DIY phantoms instead of human tissue is a clever choice, as it creates this playful and intuitive way of sculpting a soundscape, shaping the end results before they even hit an electronic translation process. It also shifts the focus to a more post-human-centered approach, in which the story isn’t as much on an individual as it is on Frankenstein-like phantoms.

If we were thinking in terms of musical instruments, the phantom in this case acts as the body of the instrument, and the probe as a tool for playing it.

On the Synthesis Approach

The paper outlines a sound synthesis framework that translates real-time ultrasound imagery into multi-spectral audio by mapping object depth (the distance from the probe’s surface) to frequency and pixel brightness (corresponding to material density) to amplitude. From my understanding, this is done via Inverse Fast Fourier Transform (IFFT) by sampling single or multiple vertical lines across the video stream. Each line is then mapped to a bank of individual frequency bins, without phase reconstruction or more advanced spectral processing. The result is a frame-by-frame additive synthesis. This approach is somewhat limiting, something the author openly acknowledges, as some of the phantom designs end up producing relatively similar soundscapes.

My first thought after reading up on ultrasound probes was to make a more straightforward synthesis approach of taking the ultrasound echo, shifting it to the audible range, applying an amplitude curve accounting for human hearing, and listening to the results. Then, I would probably build some type of detection tool for different shapes to trigger different types of sound events. From there, it could be combined with the already established idea of taking the depth and density values for some other sound parameters.

This focus on mapping dynamic frequency curves brought me back to the author’s brief, separate quick mention of the app SPEAR (an application for audio analysis, editing, and synthesis), which I would like to quickly mention as I tried using it in the past. My idea was to analyze audio samples via the software and reconstruct them via additive synthesis in SuperCollider (a platform and programming language for audio synthesis). Such an approach, however, proved more finicky than I initially expected, with the number of various sinusoidal frequencies needed to get a hint of the original sample being larger than anticipated. If I were to try again, I would also take into consideration the varying amplitude of the frequencies over time, which would be achieved by putting the needed values in arrays read as values for ADSR (attack, delay, sustain, release) envelopes.

Final thoughts

I enjoyed reading about this project. I was particularly drawn to the sculptural aspect and design choices which were taken when tackling the DIY phantoms. The project works well for a performance, as it has an anchoring point most of us would be able to understand, that being the live visual representation of the scan in combination with the exploration and manipulation of the chosen phantoms. If this were developed further, I would be interested in what other synthesis methods they would come up with, or in what way they could enrich the possibilities of the current setup.

Sources: