I chose this article because the title immediately caught my attention. I was curious what “Tiny Touch Instruments” actually are and what kinds of decisions and thoughts go into programming such instruments.
The author, Rebecca Abraham, is a researcher and composer working in the area of digital and collaborative music-making. In the paper, Abraham describes a project centered on Tiny Touch Instruments (TTIs), a set of mobile, web-based musical instruments that are played through touch gestures on a smartphone or similar device. The project is situated within a broader context of mobile music ensembles, such as the Stanford Mobile Orchestra, which explore how mobile technology can support collective music-making. You can access the Tiny Touch Instruments here.
As part of this research, Abraham composed two pieces titled Skating and Skipping. Both works are performed using the TTIs that the author programmed. The instruments run on a webpage and are controlled using gestures such as tapping, swiping, or holding a finger on the screen. These interactions generate sound while also producing visual feedback, allowing performers to see and hear the effects of their gestures. One important aspect of the project is accessibility: the pieces are designed so that they can be performed without prior rehearsal and even by people without formal musical training.
The two compositions use different approaches to notation and performance. In Skating, performers follow a graphic score that includes visual shapes and brief text instructions. Participants draw certain gestures on their screens or interact with others in the group, for example by imitating nearby performers or responding to sounds they hear in the room. The focus of the piece is less on precise melodies and more on shared sonic textures that emerge through group interaction.
Skipping uses a different format. Instead of a static graphic score, performers follow an animated score projected on a large screen. This score combines graphics, animations, and text instructions that guide the performers’ actions over time—for example, indicating where on the phone screen to interact or encouraging them to increase the frequency of tapping. The piece gradually shifts from simple exploration of the instruments toward more intentional interaction between performers.
Through observations, interviews, and surveys with participants across several performances, Abraham analyzed how people experienced these pieces. One key finding was that performing without rehearsal encouraged exploration and experimentation. At the same time, performers gradually became more comfortable with the instruments as the piece progressed. Another important result concerns notation: a multimodal approach that combines graphics, animation, and text proved particularly effective.
Visual elements helped performers understand the relationship between their gestures and the sounds produced by the instruments.
An especially interesting observation was how the performances changed participants’ perception of their smartphones. During the performance, the phone was no longer experienced primarily as a device for communication or distraction, but rather as a creative musical tool that enabled collective expression.
These ideas resonate strongly with my own design interests. In my research, I am exploring the concept of “ear candy” and interactive sound design. Inspired by this article, I am considering developing my own small touch-based digital instruments that people could access online. My goal would be to design them in a way that is not only playful and engaging, but also educational, allowing users to learn something about sound or interaction through experimentation.
Author: melina.trummer
Trying to set SMART goals
To achieve a big project, it must be separated into little, actionable steps. This blog entry refines my intention with the project further and talks more about the specific goals.
The core purpose of my project remains unchanged: I want to explore how sound design can function as a storytelling tool within a musical composition. At the center of this exploration is an element that is ever-present in pop music, yet rarely analyzed in depth: ear candy. I want to understand how these sonic details can create meaning and support a narrative—and how I can use this knowledge purposefully in my own productions.
In short: I want to treat sound design not as a technical afterthought, but as a dramaturgical element that actively shapes a song’s story.
Goals and subtasks
Using the SMART criteria for goal setting (meaning a goal needs to be specific, measurable, achievable, relevant, and time-bound) I am setting the following goals for the running and upcoming semesters:
Winter Semester 2025: Literature Review and analysis of two pop songs
| Specific | Literature review · Collect academic articles on sound design, music production, and related terminology· Collect non-academic sources (YouTube tutorials, blogs, producer breakdowns) · Explore psychoacoustics, synesthesia, and musical chills Analyze the terminology · Gather definitions, synonyms, and varying understandings of “ear candy.” · Identify recurring themes: purpose, emotional impact, sonic characteristics Analyze two pop songs · Look for elements that stand out and describe why they stand out · Analyze which effects were used and what purpose they served in the songs Start the adjective collection (ongoing across all semesters) · Note adjectives encountered in literature, interviews, and analyses · Categorize them (e.g., emotional, spatial, timbral, narrative) · Record preliminary meanings, examples, and contexts Optional: Hold an interview with a pop music producer · Find an interview partner (Research local or online producers experienced with ear candy and evaluate their relevance to the project (genre experience, professional background)) · Organize the interview · Prepare the interview questions (e.g. How do they incorporate ear candy in their work? What techniques do they use? Topics to cover: personal definition of ear candy, creative process, sound sourcing and recording, editing methods, dramaturgical intentions, best-practice examples) · Conduct the interview · Transcribe (parts of the) interview |
| Measurable | Literature Review: Finish the expose at the end of the semester and present my results Analyze terminology: Present a definition that matches the way ear candy is described in the research Analyze two pop songs: Present a summary of the findings Interview (optional): Hold the interview and summarize the findings |
| Achievable | Tasks rely on available resources (literature, online material, accessible industry contacts). |
| Relevant | Defining the term is relevant for further semesters, to have a shared understanding what the project is all about. |
| Time-Bound | Finish the literature review before New Years. Finish the presentation of the results when it is due in January of 2026. |
Summer Semester 2026: Apply ear candy concepts to an original song
| Specific | 1. Develop the base composition Create a song structure (intro, verse, pre-chorus, chorus, bridge). Produce the foundational arrangement: – chords / harmonic progression – melody or topline – percussion – bass – vocals Create a clean production without any ear candy, to serve as Version A. 2. Design & integrate ear candy elements Based on Semester 1 findings, design ear candy that: – supports the song’s story – enhances emotional impact – reinforces transitions or dynamic shifts – adds subtle textural detail Create Version B (extensive ear candy) using the full palette of techniques. 3. Listener perception study (Optional but valuable) Conduct a small informal test with listeners (friends, classmates, musicians) Ask participants to compare Versions A and B. Collect data on: – perceived narrative clarity – emotional response – sense of depth and – detail memorability Document results for inclusion in your final report. 5. Continue expanding the adjective list Add descriptors derived from the production process. Start grouping words into categories. |
| Measurable | One full song production in two versions (with and without ear candy). |
| Achievable | Uses skills developed from research and accessible production tools. |
| Relevant | Demonstrates the practical value of the theoretical framework. |
| Time-Bound | To be completed by the end of Summer Semester 2026. |
It is not completely clear to me what I will do in the third semester. I assume that producing the songs might be a process that also extends into the third semester. Nevertheless, at the end of the project I want to have achieved the following:
Specific: Compile results, finalize the adjective list, evaluate the production outcomes, and prepare final documentation
Measurable: Final project portfolio including written thesis chapter, song versions, diagrams, and terminology list
Achievable: Builds on completed work from previous semesters
Relevant: Completes the research cycle and demonstrates academic + artistic learning outcomes
Time-bound: End of Winter Semester 2026/27
Ear candy & storytelling with Sound Design
I want to explore Sound Design as a storytelling tool within a musical composition. The focus of this project is on musical details that are added to a song to make it more interesting. “Ear candy“ is a vague term used to describe sound effects, extra layers, rhythmic variations and other techniques which are supposed to make a song more interesting for the listeners. They can help to shape the soundscape and character of a song.
This project is not only about adding details to a song, but also about building a vocabulary for describing sound characteristics. The practical part of this project aims to combine songwriting and sound editing. The goal is the concept of a song that is enhanced with sound design elements.
Examples of ear candy
- The vocals in the song “Who let the dogs out” by Baha Men imitate the sound of a dog barking.
- Billie Eilish and her music producer Finneas O’Connell include sounds they recorded in real life as samples in their songs. One song by them features the drill of a dentist’s equipment, another one includes the sound of an Accessible Pedestrian Signal (APS). (APS are devices which are designed to help pedestrians who are blind or have low vision by providing non-visual cues like sounds and vibrations to indicate when it is safe to cross the road.)
Purpose of the project
The core purpose of my project is to analyze sounds, develop a vocabulary for my future sound design work and explore different ways to use sound design elements in musical compositions. Instead of treating sound design as a decorative or purely technical layer, I intend to explore it as a meaning-bearing, dramaturgical element.
Ultimately, the goal is not simply to produce a song but to research and articulate a method for using sound design as narrative strategy in music.
I want to
… compose a musical piece in which sound design shapes the narrative structure.
… create ear candy elements that contribute not only sonically but symbolically.
… develop a vocabulary to describe sound qualities and emotional functions in production.
… connect theory and practice when it comes to musical storytelling, sound design and sound editing.
Although the outcome is a standalone piece, the underlying methodology has broader applications:
· Artistic practice: designing sonic narratives, creating atmosphere and emotional depth in music.
· Music production: helping artists articulate a “sound world”.
· Education & analysis: building tools for describing sound qualities and production strategies.
Possible techniques & methods
To narrow down the project, I might draw from several categories of methods. I hereby note that these categories were suggested to me by the LLM ChatGPT after several prompts telling the model which areas of Sound Design I would like to explore more.
A. Analytical Techniques
· Harmonic analysis of emotion-driven chord progressions
· Textural and spectral analysis of reference tracks
· Identification of “Ear Candy” categories (e.g. foley events, sound effects etc.)
· Development of descriptive vocabulary for sound qualities
B. Sound Design Techniques
· Foley-based sampling (recording small sound events)
· Textural layering
· Creative vocal processing (pitch, formant, spatial modulation)
· Unusual chord voicings or progression loops
· Spatial mixing techniques for headphones (binaural cues, intimate panning)
C. Production & Composition Methods
· Writing a narrative arc for the song
· Mapping sound events to emotional or symbolic moments
· Iterative prototyping: composing → sound-designing → analyzing → refining
· Mixing for headphone emphasis
· Evaluating how well each sound supports the story
Questions that will most likely occur during this project
· How much Foley work is necessary to support the story of a song?
· Which sound design elements risk becoming too literal or distracting?
· How can I balance songwriting and sound design so that neither overwhelms the other?
A self-introducing monologue
Hi, my name is Melina and I grew up in a village, which is a little fun fact about me that will become important later.
Let’s get through the rather boring stuff first, so we have that out of the way.
I did my A-Levels at HLW Mureck, back when the school focused on gastronomy and economics. After working as a waitress, I switched industries and worked at an IT company in customer support for four and a half years. I wanted to achieve bigger things, so I applied for a study program at university. Thanks to the Austrian self-supporter scholarship I was able to study my dream at FH Joanneum. When I started to study Information Design, my dream was to be an UX/UI designer (that is somebody that thinks about how interface elements of websites and apps are positioned and styled in a way that people can use the website/app/service and have a good time doing so). However, my goal changed over time.
Long before I even applied at FH Joanneum, I discovered my love for Sound Design without knowing it was Sound Design. Growing up, I was always interested in (listening to) music. In 2009, Adam Young caught my attention with his electronic/alternative music project Owl City. Apart from his most famous song Fireflies, the song Vanilla Twilight had a lasting impact on me. I loved the way his vocals were tuned, the ear candy melodies, how the lyrics fit into the rhythm of the song and so forth. Unfortunately, when you are a teenager in a village, your everyday, real-life music opportunities are limited to a brass band or a church choir. But participating in those wasn’t my thing; I was into video games and singing along to the pop songs on the radio.
This attitude made me miss the train to music education town, figuratively speaking. To this day I can’t sight read and my theoretical music knowledge is limited.
In my early teenage years, I was faced with a huge emotional challenge. I was not able to talk about my struggles with anyone and couldn’t quite find a song that described how I was feeling. Out of the need for comfort I picked up a guitar and just started playing some random notes. Exploring what sounds good distracted me from the pain I was going through while it simultaneously offered me a way to express myself without talking. I was used to writing poems or diary entries, but songs? Unimaginable at first, but it soon happened naturally. Exploring and documenting my life by improvising melodies and lyrics over chord progressions has become my favorite thing in the world. In the past 12 years of my life, I’ve written over 300 songs (some better than others, none of them released yet).
When I moved to Graz in 2018 I bought my first own keyboard, which happened to be a MIDI-keyboard. Since a MIDI-keyboard is just a controller and not a standalone instrument, I had to install a DAW to play on it. That’s when everything changed. I was stunned by the fact there is an application that can help me make music on my computer, just like my idol Adam Young!! 😀
Moving to Graz also brought other possibilities into my life. I found some music-loving individuals and we formed an a cappella group. Unfortunately, Covid-19 made it impossible to practice, so the group split up again.
In 2021 a friend sent me a link to an open mic event. Since I mostly write sad, calm Indie-Folk songs, I took the opportunity and started to perform my songs live at that monthly event. I still perform there whenever I get the chance to, usually accompanying myself on the guitar, piano or playing together with an instrumentalist.
During my Information Design studies, I felt the most joy when experimenting with sounds. One of my favorite projects was for the Klanglicht-Festival in 2023: I had the honor to compose a jingle for their audio guide.
I believe that the skills I learn through Sound Design can help me be a better, more expressive artist. The Sound Design master’s program is my chance to take my time to deep dive into some aspects of shaping sounds.
In an ideal world I am not only shaping sounds, but also shaping experiences that let people interact or enjoy sound in an immersive way. I want to create something that lets them shut down the world around them. I would love to participate in a sound installation in a museum, or at the Klanglicht-Festival. Since I want to improve my music production skills, producing an EP would also be a project I would love to do. From an information/graphic designer perspective, enhancing (or manipulating) visuals with sounds would be interesting as well. So far my work has only contained short and simple projects, like jingles, ringtones and sound experiments. My portfolio page covers most of the projects I have done so far: https://www.melinatrummer.com/
I must consider that I have neither a usual music background, nor a technical one. So, I might not be able to do a crazy coding project or compose a 12-track album just yet. Sound Design is quite a challenge for me, but I am looking forward to learning a lot about this discipline in the next two years and onward.
All I know now is that I would love to make a project that provides people with the same level of comfort that music has brought to me all my life.