The paper introduces the concept of ViVo, a tool for instrument learning, that combines the already well-researched but until now separately used methods of vocalization and visualization. It has two modes: One that visualizes the real-time vocalizations of the user, transforming it into illuminations of the corresponding piano keys and a mode where the user can practise with visualizations of pre-recorded vocalizations.

In the introduction the author gives information about the technique of vocalizing, which is an important tool to learn instruments and improve the general sense of music and improvisation and composition skills. Its effectiveness can be attributed to the fact that vocalized melodies can be memorized better than instrumental melodies. Despite this, vocalization is not used in instrument learning environments to its full potential. Visualization, on the other hand, is broadly used in music education, guiding the learners through real-time instructions and feedback. ViVo aims to combine both learning methods to create a more effective and intuitive tool.

The tool consists of a microphone that’s being used to record the vocalizations, a processor and an electronic piano with illuminable keys. In another section there is more specific information on the used hardware.

The goals of ViVo are described as follows: portability, low material costs, effective real-time operation, pitch accuracy, pitch range and note onset/offset fidelity.

The author then describes the algorithms, that were used for ViVo: firstly, two pitch detection methods (YIN algorithm and an FFT-based heuristic approach) to extract pitch from vocal input. While standard interpolation techniques (like parabolic fitting) are already known, ViVo adapts them specifically for singing. Its modified method focuses on a typical vocal pitch range (100–1000 Hz), calculates the average energy within that range, and then scans for the first three consecutive frequency bins exceeding this average. It applies a parabola fit to these bins to estimate the fundamental frequency (f₀), optionally smoothing the result with a filter. These adjustments improve robustness to noise and reduce common errors, such as detecting the wrong octave when harmonics are stronger than the fundamental frequency.

To test if ViVo fulfils the previously set goals, it was being tested by playing a voices library through a speaker placed 70cm from the microphone to simulate the expected use conditions.

In the following sections the methods of pitch detection, range and note onset/ offset are described in more detail. After this the author gives an outlook on future work. Planned features are for example the extension of the detection system to recognize polyphonic melodies which allows to use recorded music instead of only relying on unison vocals. The use case will be extended to larger music education contexts such as classrooms. There will also be studies on how the tool can help to learn jazz improvisation. Another mentioned step is to develop the ViVo tool for other instruments such as guitar. Then the author illustrates how the tool could be used in live performances.

In the last part of the paper there is a section on ethical standards. The author declares that she used low-cost materials and open-source software that was funded by herself. She also indicates that there were no conflicts of interest and that all participation in the project was voluntary.  This part is followed by the references.

I found the approach of combining vocalization and visualization as learning methods very interesting. The prototype shown on a picture seems a bit raw and unfinished, but I think has potential to be further developed. I wonder if the idea of this tool can also be applied to other instruments that do not have a visible scale, such as trumpet, saxophone or French horn. In my own experience of learning the flute, trumpet, French horn and guitar I never worked with visualization, but sometimes used vocalization with my French horn teacher, which I think is helpful to establish a feeling for pitch and intervals and makes it easier to hit the right notes. This is especially crucial for brass instrument players, since those instruments usually only have 3-4 keys, so the same key combinations are being used for various notes that have to be controlled through air pressure.

The paper itself was well structured, although I would have put some of the sections in a different orde. In my opinion it would make more sense to put the goals of the tool right after the introduction that describes the concept of the tool. Other than that I think it was easy to understand the idea and how ViVo works.

In the second part of my blog series from the IRCAM Forum, I will summarize 2 workshops and performances, where I found similarities and approaches that could be interesting for my own research project.

Concrete Motion

Concrete Motion is an experimental tool designed for educational settings that facilitates the study and creation of sound-based music through physical movement. The system integrates the flexible audio processing of Max/MSP combined with Google MediaPipe body-tracking, within the TouchDesigner environment. By leveraging these technologies, it establishes an interactive digital space where listening and electroacoustic analysis are mediated through the user’s bodily gestures. This approach aims to bridge the gap between abstract musical concepts and tangible, physical interaction for learners and creators alike. While the objectives in this project’s research lies in educational ambitions, the fact it is using gestural interaction to create an accessible environment, can be directly translated into my own idea of an accessible interface for my project. A main difference in the technical construction would be, that the MediaPipe Hand Landmarker is running directly through Touchdesigner. I am not sure if there is an additional latency involved or if this could also be an idea for my project. Especially if I plan to get some additional visualisation on the gestures and processed audio.

Liminal

Liminal is an interactive installation that moves beyond traditional control-based models to explore a “liminal” space where agency is shared between humans and AI. In this environment, human gestures do not function as direct commands but instead serve as contextual information that influences the system’s evolving behavior over time. The architecture uses real-time computer vision and a Python-based decision layer to ensure that audiovisual changes emerge through gradual modulation and probabilistic weighting rather than immediate cause-and-effect. By distributing authorship across both the participant and the machine, the work transforms interaction into a sustained, meditative dialogue shaped by accumulated history and continuous negotiation. I visited the performance and workshop, as this is as well a similar approach from a gestural interaction to audio creation philosophy. In comparison to other projects of this theme, I found the bidirectional interaction and decision making between the human input and the models system. Maybe this could also be seen as an anecdote to how future work will, if not already, operate in all daily activities. On a technical level, the computer vision was also implemented through Touchdesigner, as in the other project mentioned earlier. 

Both projects incorporate most technicalities I strive to achieve in my own project, hence why it was interesting listening to the approaches and talking about the ideas involved. While I will head towards a different end goal with my product, they are still good examples to compare the technical feasibility and workflow. All in all, our days at IRCAM Forum brought me exciting insights and takeaways from diverse fields of audio focussed research.

Last week we attended the IRCAM Forum in Paris as part of our semester excursion. Overall it was an interesting couple of days, where we had the chance to experience a lot of the latest research and techniques in various fields of audio and sound. Within this post I will try to summarise my main takeaways on some chosen talks, workshops and performances I have attended.

Partiels & ASAP Plugins

I visited a talk and workshop by Pierre Guillot, which showed updates and news from the tools Partiels and ASAP. Partiels is a software dedicated to analysing audio signals and retrieving useful data for signal processing and sound design applications. One of the new interesting developments involved a direct python integration to access analysed data directly. The ASAP tools are more a direct creative solution to manipulate audio in various applications. The three major tools mentioned would be the Psycho Filter – allowing to apply spectral filters directly to a source, the Pitches Brew – an advanced pitch and formant manipulation via interactive frequency curve editing, and Stretch Life – a time manipulation tool for compressing and stretching sound dynamically. Further notable mentions have been the Spectral Morphing and Spectral Crossing tool, which allows to combine and ‘morph’ two audio sources on the spectral domain. All together these seem to be interesting tools, cleverly designed and quite accessible for most users I would imagine.

GRM Tools – Atelier

Another interesting workshop was the presentation of the GRM Tools Atelier software. It is a sound processing and synthesis environment working for real-time and multichannel productions, both in a live or studio application. I really liked the modular approach and the quick and intuitive randomisation capabilities, which allows for a fast agitation of multiple parameters at once. This can be an interesting choice for sound artists wanting to work with only one standalone software and dealing with more graphically intuitive controls than for example puredata or Max/Msp. As I already own some similar synthesizer software of a similar modular system, without the multichannel capabilities, I will for now stick to these though.

VASE

I also experienced a performance of VASE by composer Yuval Seeberger. The device is a motorized music-box installation that performs a 12-minute, semi-algorithmic composition by integrating a physical punched paper score with advanced digital processing. The system utilizes a specialized “ensemble” consisting of an acoustic-mechanical music-box, an analog motor, Max/MSP synthesis, and RAVE neural audio models to create a rich, layered sonic environment. While the structure follows a formal organization, the irregular communication between the computer and the mechanical hardware ensures that each loop cycle contains subtle, unpredictable variations. Through the use of Piezo and magnetic rail-coil pickups, the piece effectively bridges the gap between tactile mechanical movement and real-time AI-driven sound generation. I was quite fascinated by the various soundscapes it was able to produce and also how the random interaction by the composer influenced the installation.

A product or business idea is a structured proposal that identifies a specific problem, outlines a solution, and defines how value is created for users and stakeholders. In design-driven innovation, such ideas are grounded in real user needs and aim to create both functional and experiential improvements.

Understanding the underlying idea of a product is the first and most important step in its development. For the idea of a guiding system at German train stations the exact paraments for the final product are not yet defined. But a closer look at the product idea is still a valuable step towards more clarity and understanding.

The core problem lies in the current experience of train platforms, which are often perceived as stressful, unorganized, and confusing environments. Boarding and exiting trains can be physically demanding, especially during peak times or for individuals with limited mobility. This creates friction in the interaction between passengers, trains, and the platform itself, ultimately reducing the overall quality of the travel experience.

Addressing this issue matters because improving the usability and comfort of train travel can make it a more attractive mode of transportation. A better experience could encourage more people to choose trains over cars, contributing to reduced traffic congestion and lower environmental impact.

The proposed solution is a physical guiding system integrated directly into train platforms. While still in development, the current idea is the use of light-based elements, such as illuminated pathways, signals, or dynamic indicators, to guide passengers intuitively. This system would enhance orientation, communicate real-time information, and support smoother boarding and alighting processes without adding visual clutter.

The target audience includes all users of the train system, with a primary focus on passengers. At the same time, organizations like Deutsche Bahn act as key stakeholders and customers, investing in and maintaining the system. The expected impact includes improved navigation, more efficient passenger flow, and a more structured and user-friendly platform environment.

From a business perspective, the model could involve an initial infrastructure investment by railway operators, followed by ongoing maintenance.

Ultimately, the idea combines user-centered design with systemic impact, aiming to transform train platforms into more intuitive, accessible, and enjoyable spaces.

The Value Proposition Canvas is a strategic tool used in design and innovation to ensure that a product or service aligns closely with user needs. It consists of two main components: the Customer Profile and the Value Map. The Customer Profile focuses on understanding the user by identifying their jobs (what they want to achieve), pains (challenges or frustrations), and gains (desired outcomes or benefits). The Value Map, on the other hand, outlines how a product or service responds to these needs through products and services, pain relievers, and gain creators. Together, these tools help designers create solutions that are both relevant and impactful. (Strategyzer, 2026)

To get a better understanding of the anticipated product and its purpose for the user, two canvases were produced for two different players. The first one focuses on the train passenger as an end user. Their Customer Profile emphasizes practical goals such as arriving on time, navigating platforms easily, and boarding trains without stress. Gains include comfort, clarity, and reliability, while pains involve confusion, overcrowding, physical strain, and lack of accessible information. The Value Map responds with a physical support and guidance system, clearer information structures, and inclusive design features to accommodate diverse user needs.

The second example represents the Deutsche Bahn (DB) as a customer. Here, the Customer Profile highlights organizational goals such as transporting passengers efficiently from A to B, ensuring smooth system operations, and maintaining profitability. The identified gains include improved punctuality, enhanced public image, and increased customer satisfaction. However, DB also faces significant pains, such as technical failures, delays, and negative public perception. The corresponding Value Map proposes solutions like improved guidance systems, better information displays, and more structured platforms, all aimed at reducing inefficiencies and enhancing the overall service experience.

Overall, these two profiles demonstrate how the Value Proposition Canvas can bridge organizational objectives and user experiences, enabling more targeted and user-centered design solutions.

The “Harmonix Series” by Wing Hei Cheryl Hui and Patrick Hartono represents a visionary bridge between human-computer interaction and therapeutic art, moving beyond the technical novelty of Digital Musical Instruments to address a profound need for the democratization of creativity. What I find most compelling about this work is its commitment to radical accessibility; by shifting the focus from mastering a complex tool to simply experiencing a soundscape, the authors empower users of all physical and cognitive abilities to become creators. This is further elevated by the intentional integration of mindfulness into the interface design, which transforms the act of music making into a meditative process for emotional regulation rather than just a performance. The synergy between robust technical implementation and a sophisticated aesthetic sensibility is palpable, resulting in an instrument that doesn’t just function, but truly resonates on a human level. Ultimately, this project serves as a vital reminder that the future of music technology should prioritize deep human impact and digital wellbeing, treating the user as a whole person seeking connection and calm.