by Indigo Tschurtschenthaler -

ClimaSynth: Enhancing Environmental Perception through Climate Change Sonic Interaction

In this entry I will be reviewing a NIME (New Interfaces for Musical Expression) paper I read, ClimaSynth: Enhancing Environmental Perception through Climate Change Sonic Interaction. The authors are Eleni-Ira Panourgia and Angela Brennecke from Film University Babelsberg KONRAD WOLF and Bela Usabaev from the Cologne Academy of Media Arts. The paper was published in 2024 from the International Conference on New Interfaces for Musical Expression.

The document describes ClimaSynth, an interactive application that aims to communicate climate change through sound, using interaction as a means to raise awareness about the changing environment. ClimaSynth explores the relationship between acoustic and climatic effects by allowing users to manipulate sound through an interface.

I believe it is relevant for New Interfaces for Musical Expression because it explores how we can use simple sonic parameters to communicate big issues. At the same time, it is relevant for my research because it shows a way to draw attention to an environmental cause.

The web-based application developed wishes to enhance environmental perception and explore how sound can express aridity and drought. It achieves this through contrast and familiar sounds, which are associated with climate conditions. The aim is also to investigate the issue through non-human perspectives.

The sonic experience wants to be accessible, with a blurred distinction between content and interface. The authors wanted to create a minimal UI that united sound and vision. Other applications in this format already exist, such as Sound Canvas, Lines and GrainTrain. Riverssounds, another one, allows users to navigate river ecosystems with mouse interaction.

Climate change also means that environments sound differently. This interactive application tries to communicate this using a sound design technique called granular synthesis. It transforms audio by breaking it down into microscopic fragments called “grains”. In this case, granular synthesis is used to manipulate recordings. 

The user can select a soundscape from a drop-down menu: “birds near water”, “river water” and “tree bark”. Each of these corresponds to a field recording. They can then adjust two parameters with sliders, “areas” and “spread”, which are then translated visually and sonically. By modifying these parameters, it is possible to transition between two extremes: the birds singing alone or accompanied by insects, the river becoming dryer and the tree bark readjusting its flexibility, all changes caused by drought. 

The “spread” value is visually translated as a purple area around the mouse, that can be more or less dense. With zero spread the grains are sampled from the same point, while a higher value allows for a broader selection of grains.

The “area” value shows the number of black dots around the mouse. Lower values allow the sounds to be individually perceived, while higher values make us perceive them as connected.

The conceptual originality of ClimaSynth is using parameters to communicate climate change and identifying which ones convey the idea the best. The non-human perspective of the applications makes the user almost feel like they are a fish in the river or the tree itself.

The idea of melting interface and content is coherent and shows that sound and visual elements can also communicate greater problems without needing any words.

The paper is missing information about target audience, testing and participants.

I also think that it is not very accessible to someone not from the field because the way the parameters work is not intuitive. Their visual translation is clear, but I found the sonic changes difficult to grasp. Before reading the paper, I was not sure of what the interactions meant. I would have wished for clearer naming of the parameters or labels on the two ends of the sliders, as a sort of cue of their effect on the sound. 

With the climate changes not being too evident on screen or sonically, I think the result is limited, although after playing with it for a while it is possible to perceive the changes.

As a consequence, I believe that the mapping could be more thoughtful, but the instrument is still performable beyond the lab. It explores musical expression in a way that is closer to nature and it surely conveys a message.
I agree with what is stated in the conclusion of the paper: “This approach offers a promising direction for conveying the complexity of climate change through rich sonic encounters with changing environmental states.” 

The conclusion also mentions the possibility to integrate more parameters and environmental or location-specific data in the UI, and I can imagine the interface changing for each recording, to make the experience more immersive.

I reckon the application could be displayed in exhibitions about environmental issues, but it could also be used to promote behavioural change in interactive ads.

The method explored has potential to communicate the complex issue that is climate change, and it was of great inspiration for my research theme, as I am interested in how interaction can raise awareness about light pollution. Using parameters, whether visual or sonic and analysing them is a way to communicate environmental changes and I will take it into consideration when working on my light pollution project.

by anja.kljajic -

D&R2 SED D2 – Business Idea 2/2

What Problem are you solving?

Most game interfaces are static and cluttered, creating massive legibility and navigation barriers for visually impaired players. Designers often lack the time or specialized tools to implement complex WCAG-like standards and multi-sensory feedback.

Why should we care about it?

Accessibility isn’t just a “feature”, it is a fundamental right to play. When games like Black Myth: Wukong launch with unreadable text, it excludes millions of players and hurts the game’s reputation and reach.

What is the solution? How does it work?

An interactive design engine that acts as a “Canva for Game UI.” It allows designers to drag-and-drop modular HUD elements and automatically checks them against accessibility rules while providing a library of haptic and audio logic.

Who is the target audience/customer?

The target audience is the visually impaired gaming community who needs better tools to play. The paying customer is the UX/UI Designer and Game Studio looking to streamline their workflow and meet professional standards.

What is going to happen? (Change & Impact)

We will move from a “one-size-fits-all” UI to a Modular Era where games are playable by everyone from day one. This shift reduces “design guesswork” for studios and replaces frustration with independence and mastery for players.

Bonus: How can this make money?

The platform will operate on a Freemium SaaS model, offering a free basic toolkit for students and indies, while charging Enterprise Subscription fees to large studios for advanced simulation tools and custom haptic libraries.

by julia🐨razumovskaia -

5# Product & Business Idea – Design & Research II (Katerina)

✍️ What problem are you solving?

Most navigation systems are designed solely for speed and efficiency. They guide people from point A to point B as quickly as possible, but ignore their mood, context, and the quality of the journey. This creates a sense of passivity, stress, and detachment from the city itself, especially when it comes to large cities or complex journeys.

✍️ Why should we care about it?

Because the way we move around cities affects our attention, stress levels, and attitude toward urban spaces. When navigation focuses solely on speed, people miss out on the city’s opportunities and the chance to explore it. In large cities, this often means we stick to our usual routines and never get to know the city beyond what is strictly necessary.

✍️ What is the solution you are offering? How does it work?

The solution is an adaptive map navigation system that suggests routes based not only on the destination but also on the user’s mood, time of day, and preferences—in other words, on the right kind of onboarding. Instead of always suggesting the fastest route, it can offer alternatives such as peaceful, curious, social, or exploratory paths—for example, through parks, cafes, quiet streets, or unfamiliar neighborhoods. Safety and lighting can also be taken into account.

✍️ Who is the target audience? Who is the customer?

The target audience includes city residents, travelers, immigrants!, researchers, and people seeking more meaningful ways to navigate urban spaces. This is particularly relevant for users who are tired of navigation focused solely on efficiency (such as myself). Clients may include individual users as well as city tourism organizations, municipalities, or mobility-related services interested in promoting a more measured and people-centered urban experience.

✍️ What is going to happen? (Change & Impact)

My product prioritizes mindful navigation over mere ease of use. Instead of simply speeding up movement, users get more information, more choices, and a deeper connection to the city. This can reduce stress, encourage exploration, and help people discover new neighborhoods, rhythms, and everyday experiences in places they already know—or think they know.

Thanks! 🙏

by julia🐨razumovskaia -

4# Customer Profile & Value Proposition Map – Design & Research II (Katerina)

💼 Persona I – Busy Commuter

This is a time-conscious person who needs to move efficiently between locations, often under stressful conditions. They rely on navigation to save time, but in doing so experience stress, overload, and reduced alertness. They need routes that remain efficient, reduce mental strain, and allow for brief moments of calm.

🗺️ Persona II – Curious Explorer

A user who explores the city with an open mind and curiosity, rather than in a hurry. They are interested in discovering new places, atmospheres, and experiences. They need a navigation system that supports exploration, suggests meaningful routes, and helps them engage more deeply with the city.

by julia🐨razumovskaia -

NIME Article Review – Sonic City: The Urban Environment as a Musical Interface

I’d been looking for a topic that interested me for a while—and the title itself caught my eye because I was searching by keywords. Of course, I’m not usually interested in much of the technical information related to “music.” But this really caught my attention.

Sonic City is a powerful and innovative project precisely because it presents the city not as some kind of “backdrop” for interaction, but as the interface itself. This certainly resonates with me as someone with a background in architecture. There, we always thought of the city in terms of movement, rhythm, route, atmosphere, and—not least—the physical perception of space. In Sonic City, all these qualities are not just characteristics of the environment, but also part of the interaction. The authors of the article literally propose “playing the city,” transforming everyday movement into a form of musical and spatial experience. This idea is explicitly stated in the project’s main text: the city is viewed as an interface, and mobility as an interaction model.

From an interaction design perspective, I find it particularly significant that the project moves away from the conventional screen-based UX. There is no traditional screen, buttons, or graphical interface here. Instead, everything is built around body movements and environmental data—rotations, movement, light, even pollution levels and the presence of metal—as well as pauses and combinations of context and behavior. The authors describe a two-level mapping system, where some parameters influence short musical events and timbre, while others affect the more structural composition of the sound. This makes the interaction less abstract and more situational; the music arises precisely from the connection between a person and a specific place and moment.

The first text “In Duet with Everyday Urban Settings: A User Study of Sonic City“ [1] struck me as a stronger conceptual and design-oriented exploration of the questions ”Why can the city be an interface?“ or ”How does this project differ from a musical performance?”. It was also very interesting to see how they included not only engineers and designers in their team, but also architects and sociologists. The project isn’t just about technology; it’s also about the body, the city, behavior, everyday life, and its aesthetics.

The second text “Sonic City: The Urban Environment as a Musical Interface” [2] is a bit more important to me, since it tests whether the idea works in real-world user experiences. The study was smaller and more qualitative than quantitative. It included people of different ages and lifestyles, and, by the way, none of them considered themselves musicians. This is an important point, since the project was tested on ordinary people. The authors, by the way, asked the right UX question: “Do people perceive Sonic City as a research tool or as a musical one?”

Overall, I consider Sonic City a very important example of how to think at the intersection of architecture, urban experience, and interaction design. For me, the project’s value lies precisely in the fact that it translates spatial perception into an interactive experience, and interactivity back into urban and bodily reality. The papers showcase not just interesting technology, but a different way of thinking about UX: not merely as interaction with a screen, but as interaction with space, the body, the route, and the atmosphere of the city. And that is precisely why this project still seems relevant to me today.

Sources 🛈

[1] In Duet with Everyday Urban Settings: A User Study of Sonic City. Proceedings of the 2004 Conference on New Interfaces for Musical Expression (NIME 2004). Authors: Lalya Gaye, Lars Erik Holmquist. Available at: https://www.nime.org/proceedings/2004/nime2004_161.pdf

[2] Sonic City: The Urban Environment as a Musical Interface. Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME 2003). Authors: Lars Erik Holmquist, Lalya Gaye. Available at: https://www.nime.org/proceedings/2003/nime2003_109.pdf

by julia🐨razumovskaia -

3# Inclusion & Accessibility – Design & Research II (Katerina)

These factors highlight that access to slow navigation is not the same for everyone. The experience depends on physical abilities, emotional awareness, and cognitive skills—such as the ability to walk, recognize one’s mood, and make decisions about the route. It also requires a certain level of spatial awareness, which is not always equally developed or maintained.

At the same time, accessibility is shaped by external conditions: pedestrian infrastructure, GPS accuracy, access to mobile devices, safety, and cultural habits of movement. This shows that slow navigation is not just a matter of design, but a systemic issue influenced by the environment, technology, and the social context in which the user finds themselves.

by julia🐨razumovskaia -

#2 Change & Impact – Design & Research II (Katerina)

This comparison highlights the shift from efficiency-oriented navigation to experience-oriented movement. Most existing navigation systems prioritize speed, offering the fastest route by default and guiding users through the city with minimal user involvement.

As a result, users typically follow directions passively, often losing their sense of the surrounding environment and missing opportunities to interact with it. Travel becomes functional rather than experience-driven.

In contrast, this project explores navigation as something more adaptive and human-centered—where routes respond to mood and context, encourage exploration, and foster a more meaningful relationship with the city.

by julia🐨razumovskaia -

#1 System Map – Design & Research II (Katerina)

This diagram illustrates the broader ecosystem surrounding my slow map navigation system. At the center are the direct users—people who move around the city with different needs, rhythms, and intentions. Surrounding them are groups that use maps in their daily lives, from residents to travelers, each of whom perceives the city in their own way.

The outer layer shows indirect users—entities that do not use the product itself but shape the conditions in which it exists. Urban planners, local businesses, infrastructure, and data systems—all of these influence how we move, which routes are available, and how the city is perceived.

Together, this underscores that navigation is not just a tool, but part of a larger system. How we move through the city is shaped not only by personal choice, but also by design decisions, the environment, and the social structures around us.

by martina.monterisi -

Accessibility requirements and barriers

Breaking down the accessibility requirements of this project felt like one of the more hard exercises of the whole research process, because we are used to think everything would have been “easy and smooth” since it’s a niche digital research product. But there was much more behind.

What the user should be capable of

On a physical and personal level, sight would be the most relevant sense, since the experience is largely text-based, but it can be made compatible with screen readers and text-to-speech tools from the start. Hearing is not a requirement at all. Movement needs are minimal too, limited to basic typing, clicking, or scrolling, with voice dictation and keyboard-only navigation as fallbacks.

On the cognitive level is where it gets more demanding. Following an iterative loop of prompting, reviewing, and editing requires sustained mental focus and a willingness to sit with a process that is genuinely not instantaneous. As well as the need for a strong digital literacy to navigate AI tools or terminology, and enough language confidence to work in what are predominantly English environments. Though nowadays everything can be translated in real time.

Financially, the core methodology is designed to be of course a free to access tool, to be used as a starting point or guideline to use AI tools. Infrastructure needs are also few: a device and internet connection.

Who is it meant for and where does this happen

The methodology will be designed for anyone engaging with storytelling on a professional or exploratory level, from researchers and creatives to students and private users. It lives entirely in digital space, which means it can happen in any place and environment.

What it does require is probably the mindset: willingness and awareness to co-create with a machine, and enough critical thinking to question what the machine gives back.

The barriers

Two barriers kept coming up in the entire thinking process. The first was language, since AI tools lean heavily on English, nuance is often the first casualty of translation. The solution here could be to develop a simplified and even more visual version of the methodology that relies on basic English, diagrams, and examples rather than dense theoretical language.

The second barrier was knowledge. The informatics specific terminology involved is genuinely intimidating and not of common knowledge (it was for me too). The solution I thought for this issue was to add a vocabulary directly into the framework itself, so all the basic knowledge can be in the same place as the product.

To conclude, cognitive overload is also worth a mention. When the prompting loop feels endless and the output feels overwhelming, the step-by-step structure of the methodology becomes less of a nice-to-have and more of a way to escape.