desres25.designandcommunication.net

In May, we took a day trip to Linz, a UNESCO City of Media Arts, to check out the ARS Electronica Center, an institute and creative ecosystem combining the fields of art, science, and technology. In this blog entry, I will cover some projects that caught my attention.

Biology and Art

One of my highlights was seeing the research and various approaches stemming from biology.

For example, Interwoven, an artwork by visual artist Diana Scherer. By studying plant root growth, she developed a biotechnique in the form of various templates that the root system adapts to. The result shown was an intricate, textile-like pattern. This is why I was drawn to this research, as the resulting patterns were a mix of human intervention and biological adaptivity, creating various interesting textures and shapes with naturally occurring randomness rather than digital computation.

One more textile design practice I was drawn to was Growing Patterns, Living Pigments by Julia Moser, which explores dyeing processes via pigment-producing bacteria in combination with various techniques. The bacteria used, which naturally produces a bluish-purple pigment called violacein, reveals organic patterns and hues, as well as showcase a potentially sustainable approach to textile coloration.

Both of these works are based on co-creation with non-human biology. Organoids by Jürgen Knoblich and Madeline Lancaster, in a way flips the script by using human stem cells as the starting point. Created for scientific research rather than artistic production, the project focuses on organoids, masses of cells grown from stem cells with brain-like structures similar to the embryonic brain. The project gives new insights into how the human brain is formed, which can help researchers understand the development of brain disorders.

~21,600 by Dorotea Dolinšek looks at the human through the lens of an organism defined by an intrinsic, vital dependence on air. The resulting version shown at ARS was a prototype of the work, made by blowing coal dust onto paper over the course of 9 hours, using her breath as the central gesture. This work interested me due to its results: a collection of coal dust dots, none exactly the same, creating a pattern that could be interpreted as an algorithm and potentially used for sonification and algorithmic composition. In the end, she made it into a video performance spanning 24 hours, featuring a mask with a microphone as the input and an airbrush attached to the nose of the mask as the output.

AI and Art

During our visit, we got a tour of the Piano Room, equipped with a Yamaha DC1, a grand piano capable of self-playing, attached to an AI-trained algorithm for visualization developed at Ars Electronica Futurelab. For a practical demonstration, I got to play an improvisation. It was intriguing to see the resulting visuals, as the algorithm was trained mostly on classical music. While playing a repeating phrase, it visually adapted and formed a sort of memory of the visual pattern, referencing it for a short moment afterward.

While in no way new, it was still interesting to see and interact with installations using GANs (generative adversarial networks), a generative AI framework comprised of two neural networks that compete against one another to create synthetic data.

Pix2Pix: GANgadse lets users draw sketches, which are turned into cat pictures. This was done by feeding the algorithm cat images and training it to recognize typical features, like pointy ears and cat-shaped eyes. In practice, it reads your drawn shapes and approximates a guess. This was interesting to play around with, as drawing nonsensical sketches produced abstract and slightly unsettling flesh-like objects. We also tried drawing an elephant, which it turned into a fuzzy creature, and a cat, which worked as intended.

ShadowGAN uses a camera to capture the outlines of visitors and, using the algorithm, reinterprets those silhouettes as landscapes. In this case, the AI was trained with images of mountains and their landscapes, with the aim of interpreting everything it sees through this lens. The result was an interactive installation comprised of various abstract landscapes.

Both of these projects create digital forms that adapt to incoming input, almost like digital organisms.

3D 8K Experience

Lastly, I’d like to mention Deep Space, a 3D experience space developed by the Ars Electronica Futurelab. The room consists of eight 4K projectors, able to produce 8K when paired in twos, projected onto the wall and floor to create one continuous space. This is combined with 3D shutter glasses for visitors and a laser tracking system, though in our case, the interactive tracking capability wasn’t put to full use, since we experienced the program seated rather than moving through the space ourselves. The space also features a modular multichannel audio system.

Our guided tour through the experience consisted of nautical and space themes, with the most memorable moments being a 3D projection of a whale, followed by an artistic piece consisting of 3D-projected lines engulfing the audience that act as a translation and visualization of a musical composition.

Sources:

https://ars.electronica.art/news/en/
https://ars.electronica.art/futurelab/en/projects-deep-space-evolution/
https://dianascherer.nl/
https://www.livingpigments.com/
https://ars.electronica.art/center/en/organoids/
https://www.doroteadolinsek.com/projects/21-600-2
https://ars.electronica.art/center/en/piano-room-2/
https://en.wikipedia.org/wiki/Generative_adversarial_network
https://ars.electronica.art/futurelab/en/projects-understanding-ai/
https://ars.electronica.art/center/en/pix2pix-gangadse/
https://ars.electronica.art/solutions/en/deepspace/
https://ars.electronica.art/futurelab/en/projects-deep-space-evolution/

This blog is the second of a two-part retrospective on the excursion that we, the first-year sound design students, took to Paris. The topic at hand is a few activities outside the scope of IRCAM that I would like to highlight for being memorable. If you are interested in some of the happenings at IRCAM, you can check out the previous blog.

Audio Guide Tour Through the Hôtel de la Marine

Firstly, I would like to mention the location-based audio guide at the Hôtel de la Marine, an eighteenth-century historic monument and museum overlooking Place de la Concorde in the 8th arrondissement of Paris. It uses binaural sound technology with over-ear headphones that trigger different parts of the story depending on your location within the building. The over-ear design of the headphones worked surprisingly well, with audio quality that was noticeably higher than the usual museum experience.

The audio guide covers a significant portion of the building’s history, spanning more than two centuries. Covering all of this in just two hours occasionally made the story confusing due to the amount of information and the narration jumping between various historical moments. That said, the voice acting and audio dramas were immersive and well-produced.

One moment that particularly stood out to me was when the audio guide prompted me to look through a small opening in the wooden shutters, explaining how it had been used during the German occupation of Paris. As the story unfolded, it shifted its focus to the building across the square, mentioning a Nazi flag that once flew there before moving to the moment when the French Tricolour triumphantly took its place again. Seeing the flag flying in the same position on the building across the street was an example of how the immersion and interaction of the experience can be enriched to an even greater degree.

Philharmonie de Paris Musée de la musique

Another visit I enjoyed was to the Musée de la musique at the Philharmonie de Paris, located in the 19th arrondissement. The museum houses a large collection of instruments covering several centuries and cultures. Before entering the exhibition space, we were each given an audio guide. The highlight for me was the wide range of instruments, especially those that I could hear as audio samples through the guide. We also had the opportunity to hear a live performance featuring the erhu, sometimes referred to in the Western world as the Chinese violin.

The museum’s extensive collection felt somewhat overwhelming. We spent what felt like three to four hours there, and I still was not able to properly get through all the instruments I was interested in. I was surprised by the number of instruments catalogued in the audio guide app. I especially enjoyed learning about instruments that featured both a video demonstration and a detailed description. It is a shame that not all instruments are covered to this degree, with the level of documentation varying from highly detailed to almost nonexistent.

It was also nice to see the collection of electronic instruments, although it was somewhat sparse compared to the other periods represented in the museum. The Hammond Model A organ, Minimoog Model D, and UPIC, an interesting visual music computer invented by the avant-garde composer Iannis Xenakis, stood out the most. I would also like to mention the collection of instruments featuring fantastical animals as decorative elements, such as a nineteenth-century French saxhorn with a decorative head whose tongue flails around when it is played.

We were also fortunate enough to view the reconstructed studio of the musique concrète composer Pierre Henry, which was actually closed to the public that day due to renovation. It featured several interactive modules aimed at understanding Pierre Henry’s work in a more hands-on way. My favourite was the tape machine module, which produced solid results from a simple setup.

Sound Installation at the Pleyel Footbridge

Lastly, located in Saint-Denis, a commune in the northern suburbs of Paris, we visited the Pleyel Bridge, a crossing tied to the new Saint-Denis Pleyel transit hub. Here, we viewed Promenade Sonore: Vent, Soleil, Pluie by the Swiss sound artist and landscape architect Nadine Schütz. Installed on the bridge as a collaboration with the architect and engineer Marc Mimram, as well as in part with IRCAM, the work transforms the supporting structure into sculptural sound-generating instruments, each corresponding to a meteorological element. We were able to experience two of the sonic sculptures, Soleil (sun) and Vent (wind).

Soleil uses luminosity, temperature, and the time of year to evolve its soundtracks and their effects. It consists of a steel sheet supporting structure housing 24 transducers used to resonate the surface. The sculpture invites pedestrians to press their bodies against it in order to feel and hear the vibrations of the composition. This is an element I particularly enjoyed, as it adds an interactive aspect with varying and localized sonic results.

Vent, located further along the bridge, turns wind vibrations into a musical layer. Sixteen tensioned strings are stretched across the bridge arches and connected to piezoelectric microphones, which capture the vibrations caused by the wind. This installation stood out to me due to its interaction with the trains passing below, resulting in a mixture of environmental and generated sonic content.

From my perspective, the soundscapes of the bridge do not forcefully impose themselves on the passerby but rather aim to enhance the atmosphere of the space. Viewed through this lens, the installation reflects a degree of human-centred design. Human-oriented choices also appear in the architecture of the adjacent Saint-Denis Pleyel Station. Designed by Japanese architect Kengo Kuma, the station incorporates fireproofed wooden beams throughout its structure, adding visual warmth while also functioning as acoustic treatment. Rather than allowing the harsh noise and reverberation commonly associated with concrete transit hubs to dominate the environment, the wooden elements help soften and regulate the interior soundscape.

While the station seeks to reduce and control some unwanted acoustic effects, the installation by Nadine Schütz takes the opposite approach by introducing new sonic elements into the environment. Together, they demonstrate how human-oriented acoustic design can be achieved either via reduction or via extension.

Sources: