Recently, I came across the research “Entangling with Light and Shadow: Layers of Interaction with the Pattern Organ” by Jasmine Butt, Nathan Renney, Benedict Gaster, and Maisie Palmer, developed within the Expressive Computer Interaction Research Group at UWE Bristol.
RESEARCH OBJECTIVE
Figure 1: Illustration of an interface pattern
This research explores the design and use of a camera-based digital musical instrument called the Pattern Organ. This visual-audio synthesis artifact investigates new ways of interacting through light and shadow.
Users can modify a waveform by placing their hands or objects in front of the instrument’s camera, creating shadows and patterns. Through this interaction, they can perceive how both the environment and the sound change in real time.
The project initially started as a digital tool to represent the process of optical sound technology. However, during the workshop sessions, this idea evolved further. The focus shifted from a purely visual-audio synthesis system to a more open, participatory, and exploratory process.
THEORETICAL BACKGROUND
A matter of finding the grain of the world
Bruna Goveia Da Rocha and Kristina Andersen [2]
Figure 2: Design of the original Instrument
Drawing from analogue optical sound technologies used in early cinema, the research reinterprets these practices through a post-human perspective. Two main theoretical perspectives are considered. The first is from N. Katherine Hayles, who describes the world as a complex and highly interconnected syste[3]m. In this view, cognition is not limited to humans but moves dynamically across humans, animals, and technological systems. The second perspective is Karen Barad’s Agential Realism [4]. This theory describes reality as something continuously shaped by the interaction between material and meaning. Matter and information are not separate but constantly influence each other.
A strong emphasis is also placed on material thinking and hands-on experimentation with different materials.
CONSIDERATIONS
Figure 3: Images from the first workshop’s exploration
Throughout this process, there are two aspects that I personally find particularly interesting.
From a theoretical perspective, I find the idea of an entangled and participatory workshop very powerful. In this context, three elements—human, machine, and materials—are in constant dialogue. They continuously influence each other during the creative process. This approach is very effective in stimulating critical thinking, both in design, where each input can generate new ideas or solutions, and in educational contexts.
Figure 3: Sperimentation using a rotating can
From a practical perspective, I was particularly interested in the use of raw data. This concept influenced the method of sonification used in the project. Rawness can be understood as a choice to avoid interpreting or transforming the data through complex digital processing. Instead, the data produced by the system is used more directly, without adding layers of interpretation.
This does not necessarily mean that raw data is more accurate or more realistic. Rather, it means that the measurements are not modified or filtered, allowing a more immediate connection with the original signal.
In the case of camera-based sonification, two main approaches can be identified:
Extracting features from image data to control or modulate sound
Using a more direct method, where pixel brightness values are translated into sound signals with minimal processing
CONCLUSION
This research opens important questions about how data should be treated and interpreted. It challenges the idea that data always needs to be processed, optimized, or controlled.
It also highlights the role of human intervention and how our decisions shape the way systems behave. At the same time, it shows how the physical and material nature of interaction—light, shadow, objects—can influence digital processes in meaningful ways.
More broadly, it invites us to rethink the relationship between humans, technology, and the material world. Instead of separating them, this work suggests that meaningful interaction emerges from their continuous entanglement.
REFERENCES
[1] J. Butt, N. Renney, B. Gaster, and M. Palmer, “Entangling with light and shadow: Layers of interaction with the pattern organ,” in Proceedings of the International Conference on New Interfaces for Musical Expression (NIME ’25), Canberra, Australia, June 24–27, 2025.
[2] Bruna Goveia Da Rocha and Kristina Andersen. 2020. Becoming Travelers: Enabling the Material Drift. In Companion Publication of the 2020 ACM Design ing Interactive Systems Conference. ACM, Eindhoven Netherlands, 215–219. https://doi.org/10.1145/3393914.3395881
[3] Katherine Hayles. 2006. Unfinished Work: From Cyborg to Cognisphere. Theory, Culture Society 23 (2006), 159–166. https://doi.org/10.1177/0263276406069229
[4] Karen Barad. 2007. Meeting the universe halfway: quantum physics and the entanglement of matter and meaning. Duke University Press, Durham London.
The accessibility analysis focuses on both requirements and possible barriers.
The main requirements include a simple and intuitive interface, so that the product can be used without instructions. It should be compatible with different types of plants and require low maintenance, so it does not add extra effort for the user. A playful design is also important, especially to engage children and make the experience enjoyable. Finally, the feedback should involve more than one sense, combining light and sound to create a clearer and more inclusive interaction.
At the same time, some barriers need to be considered. For example, if the interaction relies too much on sound, it may limit accessibility for some users. Smart home devices can also be expensive, which may reduce accessibility for a wider audience. Another common issue is the lack of time people have to take care of plants.
In coclusion, the product should find a balance between these different needs.
This phase explores the impact of the product by comparing situations before and after its use.
Before using the device, plants often do not receive enough water. After using it, plants receive the right amount of water. Another issue is that users frequently forget to take care of their plants, while with the device, plant care becomes simpler and more enjoyable. Finally, it is often unclear when and how much to water plants, but with the device, users — including children — become more aware and learn how to take care of them.
However, some important considerations emerge. There are many different types of plants, and not all of them need the same amount of water. In some cases, the soil may appear dry, but the plant does not need to be watered yet. For this reason, the system should not give overly simplified feedback, but should consider different plant needs.
It is also important to understand what really motivates people. Users often struggle with consistency and knowledge rather than intention. They may want to take care of plants, but forget or feel unsure about what to do.
The device should therefore not only inform, but also support and encourage users, making plant care feel easy, clear, and rewarding.
The system map places the product at the center, but expands the perspective to a wider ecosystem. Instead of defining it as a single object, such as a lantern, the project considers a broader category: a smart plant care device. This allows more flexibility in form and use.
Around the product, the first layer includes direct users and elements directly involved in plant care. These are not only people, but also natural agents such as sunlight, soil, and water, which all influence how plants grow. Human users include generations such as Gen Z, millennials, kids, and the “indoor generation,” meaning people who spend most of their time inside and may have less direct contact with nature.
The second layer includes indirect users. These are people who may not use the device directly but are still connected to it. For example, teachers could use it in educational contexts, especially with children. Kids can interact with the device in a playful and learning-oriented way. People with visual impairments are also considered, since the use of sound can support accessibility. Baby boomers, such as parents, might not be the main users but could be interested in the product through their children.
The outer layer includes providers and institutions. These can be stores like IKEA or OBI, which are related to home products, or suppliers of electronic components such as Arduino and ESP32. Florists are also relevant, as well as museums, if the product is used in installations or educational workshops. Shops that sell lighting products are important too, since the device can function both as a practical tool and as a decorative object.
What if plants could communicate their needs in a simple and intuitive way?
This project explores how design can make plant care easier and more engaging through light and sound. Instead of checking apps or guessing, the idea is to create a product that gives immediate feedback, helping users understand their plants at a glance.
In the next blog posts, the different steps of prototyping and developing this plant care product will be presented.
Concept
The concept is a design product for plant care. When the product is brought close to a plant, it detects the soil moisture and responds by changing color to indicate the hydration level. At the same time, each plant can trigger a specific sound.
Technology and Interaction Exploration
An important part of the project is exploring how to design sensory feedback.
The idea is to combine sound and light to communicate information. Different technologies are considered and tested to find the most effective solution.
These include:
ESP32 and Arduino for system control
Infrared sensors or magnets for proximity detection
Soil moisture sensors to detect hydration levels
LED lights to display feedback through color
This phase focuses on understanding which components work best together and how they can create a smooth interaction.
LO-FI Prototyping
The project developed through different low-fidelity prototypes.
First prototype:
A schematic was created to illustrate how the system works, and a visual will be included to support it. In the poster, the provisional title “Biofeedback Garden” was introduced, together with alternative name ideas and sketches that show how the interaction takes place.
Second prototype:
The design of the product was explored. The idea is to give it the shape of a lantern, with LED lights inside that can change color. The form should feel soft and suitable for an indoor environment.
Third prototype:
A digital simulation was created using a QR code placed on a plant. When the QR code is scanned, a website opens showing the hydration level as a percentage. The background color changes to simulate the light of the product. This prototype also simulates the action of bringing a device close to the plant and the color change of the lantern.
Insights from User Testing
Some important insights emerged from a class test.
The website should not only show a percentage value, but also explain what the value means. The interface needs to be clearer and more direct.
It also became clear that using a phone is not the best solution. At home, people often want a break from technology. A physical, non-digital object is more appropriate for this context.
Finally, the color system needs improvement. The initial idea was:
Green = well hydrated
Yellow = medium
Red = needs water
However, this feels too similar to a traffic light. For indoor use, a softer and more “cozy” color palette would be more suitable. One possible solution is to allow users to choose their preferred colors.
In this latest article, we gather the key reflections that emerged on the relationship between nature, technology, and interaction design.
A Personal Paradox as a Starting Point
This research was born from a personal paradox. I’ve always loved being immersed in nature, but in everyday life, I’ve often found it difficult to care for plants. Lack of time, space, and continuity. From here, a question arose: what if technology could help us reconnect with nature, instead of distancing us from it?
The Indoor Generation and the Loss of Natural Rhythms
As the indoor generation, we spend approximately 90% of our time indoors: homes, offices, schools, and transportation. Although we often don’t realize it, our lives unfold almost entirely indoors. Yet, humans have a profound connection with the natural world. For much of our evolution, we lived outdoors, following the rhythms of light, seasons, and ecosystems. This connection has been progressively neglected in recent centuries. The concept of biophilia reminds us of this: we have a natural need to connect with life and vital processes, not just to live in efficient and comfortable environments.
Why It Matters for Interaction Design
Home automation systems today are highly advanced. They manage light, temperature, security, and energy efficiently. But they focus almost exclusively on physical comfort. These technologies don’t take into account deeper needs: emotions, perception, and a connection with time and nature. Smart homes know when to turn on a light, but they don’t know how we feel. They don’t help us build a connection with living things. This gap isn’t technical, it’s human. And this is precisely where interaction design can make a difference.
Methods
The research followed a qualitative and exploratory approach. The goal is not to measure performance, but to understand experiences, perceptions, and behaviors. The analysis focused on three key aspects:
engagement
feedback
emotional connection
Results
Three main directions emerge from the research.
The first concerns emotional connection. Interdisciplinary approaches, such as the use of sound, can make plants’ vital parameters visible. Transforming biological data into sounds or musical outputs allows for a more sensitive and empathetic relationship.
The second concerns engagement. Mechanics inspired by video games and gamification can make plant care more engaging, encouraging continuity and attention over time.
The third concerns nature-based user interfaces. Tools like Makey Makey show how natural elements can become an active part of the interaction. But above all, how it is possible to integrate biological inputs in a sensorial way.
Conclusion: From Efficiency to Well-Being
What if technology could bring nature back into our daily lives? This was the central question of the entire project. The current market certainly already offers several solutions capable of integrating some of the principles discussed, such as gamification. The next step could be to make these technologies more accessible and even more interdisciplinary. Integrating different media, such as sound, is also important because many home automation systems today are primarily voice-based and therefore inaccessible to those who cannot use their voice. But perhaps the most important question is not what technology can do for nature, but rather to continue asking what nature can teach us about how to design better interactions.
How can feedback and gamification make plant care more intuitive?
In recent years, a growing number of projects have sought to simplify complex everyday tasks by integrating new technologies into daily life.
CES (Consumer Electronics Show) is one of the most important international conferences dedicated to technological innovation. CES 2026, held in Las Vegas, showcased numerous projects related to wellness, smart homes, and sustainability. Among these, one of the most interesting was LeafyPod, a smart vase that combines technology, design, and nature.
In this article, we’ll explore why LeafyPod is a good example of how feedback and gamification can improve the plant care experience, making it more intuitive, engaging, and learning-oriented.
LeafyPod: A Smart Planter
LeafyPod is a smart pot designed to support the care of indoor plants. Through sensors that detect soil moisture, light, temperature, and environmental conditions, the system provides clear guidance on when and how to intervene. What’s unique about LeafyPod is that it doesn’t just collect data, but translates it into simple, user-friendly information. Through a dedicated app, users can identify a plant, receive personalized instructions, and track its health over time. The system uses an artificial intelligence engine that learns from the plants’ actual conditions and improves the guidance provided.
This year’s version features an expanded AI engine to support a much larger number of indoor plants. Users can search for any plant by name, explore an ever-expanding global catalog, or take a photo for instant plant identification along with personalized care instructions. [2]
Image 1. App and pot design of LeafyPod from LeafyPod.com
Feedback and gamification
One of the most interesting aspects of LeafyPod is the way it uses feedback to foster learning. The app’s notifications, alerts, and prompts transform plant care into a series of small daily goals. Gamification isn’t present in the form of explicit games, but through positive micro-interactions: reassuring messages, visible progress, and clear suggestions. This approach helps build a routine and maintain motivation, transforming plant care into a gradual and rewarding experience. In this way, LeafyPod uses feedback not only to inform, but to guide user behavior, creating continuous interaction.
Downsides of LeafyPod
Despite its innovative approach, LeafyPod also has some limitations to consider. LeafyPod isn’t cheap: individual planters cost tens of dollars, and to work properly with the app and Wi-Fi connection, you often need to purchase an additional Bridge. This can make the initial investment more expensive, especially for those with multiple plants. According to the official specifications, LeafyPod doesn’t currently integrate directly with smart home systems like Alexa, Google Home, or Apple HomeKit. This limits the possibilities for automation and integration with other devices already present in the home, although such features may be introduced in the future.
In the era of digital transformation, game design and interaction design are increasingly being integrated. Game principles are used not only to entertain, but also to enhance the user experience of everyday applications and services. The goal is to make interactions clearer, more engaging, and easier to understand, helping users participate more actively and consciously.
Core Principles of Gamification
Gamification aims to make common activities more interesting and enjoyable. To work, an app must be able to maintain the user’s attention and motivate them to continue using it. This happens when the proposed challenges are balanced with the user’s abilities, creating a sense of continuous engagement, called flow. According to Rigby and Ryan, a good gamified experience is based on three main elements: autonomy, competence, and relevance. Autonomy concerns the ability to choose and feel in control, competence is linked to improvement and the feeling of succeeding in what one does, while relevance concerns the meaning of the activity and the connection with others. [1] Another fundamental aspect is progression: the user is guided step by step through increasingly complex objectives, receiving rewards and clear feedback. Feedback must be simple, immediate, and useful, so as to help the user understand the effect of their actions. Narration, understood as a coherent theme or purpose, also contributes to making the experience more memorable.
These principles can also be intertwined with the biophilic design field. One example of these principles is Senso, a smart, gamified sensor for plant care. Senso monitors data such as soil moisture, temperature, and sunlight in real time, using artificial intelligence to provide helpful suggestions to the user. The experience is made more engaging thanks to a small pixel-art-style digital character that communicates information and guides the user in caring for the plant. This way, everyday tasks such as watering or controlling the light become more intuitive and less repetitive. Senso transforms plant care into an interactive experience, demonstrating how gamification can improve usability and engagement even in non-gaming contexts.
Image 2 – Image 3 – Image 4. Senso a gamified Plant sensor from Prelaunch
References
[1] S. Rigby and R. M. Ryan, Glued to Games: How Video Games Draw Us In and Hold Us Spellbound. Santa Barbara, CA, USA: Greenwood Press, 2011.
[2] S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From game design elements to gamefulness: Defining ‘gamification’,” in Proc. 15th Int. Academic MindTrek Conf.: Envisioning Future Media Environments, Tampere, Finland, 2011, pp. 9–15, doi: 10.1145/2181037.2181040.
Why is pothos the best plant for starting to care for plants? And why is pothos a simple and natural example of how good interaction works? In recent years, more and more people are becoming interested in the world of plants, often living in cities, in small apartments, and with little time to spare. However, caring for a plant isn’t just about adding a decorative element to the home; it’s also about starting to observe how something can change over time.
Pothos is a very simple plant to care for, making it ideal for learning. Its changes are easy to notice: the leaves become softer when thirsty, the color changes depending on the light, and growth slows or accelerates. By observing these signals, we begin to understand what the plant needs and how to respond.
This mechanism helps us understand a key principle of interaction design: feedback. Our actions produce a result that modifies a feature of the interface and allows us to understand the next steps to take. In the natural world, feedback arrives slowly, over time. It is not as immediate as a digital interface, but for this very reason it requires attention and observation skills.
Pothos: a Great Plant for Beginners
Pothos (Epipremnum aureum) is often recommended as a first plant, and for good reason. It is a tropical climbing plant, capable of adapting to a wide range of conditions. It thrives in both bright light and partial shade, requires little watering, and survives even minor care mistakes. [1]
But its real strength is how easy it is to understand its needs. When thirsty, the leaves lose vigor. When too much light hits, the color changes. When healthy, it grows rapidly.
Looking at pothos, we learn that good feedback doesn’t have to be complicated. It just needs to be visible and consistent. This form of slow, natural interaction helps us understand how, even in the design of digital interfaces, feedback is essential for building intuitive, accessible, and learning-oriented experiences.
Slow Feedback and Interaction Design
In the field of interaction design, one of the fundamental principles is feedback. An interaction works when the system responds to the user’s actions in a legible and coherent way. The pothos works exactly like this. Humans act, the plant responds over time. This relationship isn’t as immediate as the digital one, but for this very reason, it’s educational. It teaches us to recognize patterns and respect timing.
Nature-Based User Interfaces: The Makey Makey Example
In recent years, interaction design has begun to explore nature-based user interfaces, or interfaces that use natural elements as inputs, outputs, or communication mediums. Among the various examples is Makey Makey—a sensor that, like a keyboard or mouse, can become an input for the computer. It is a system designed to create tangible interfaces simply and immediately, without the need for programming or building complex circuits. Its unique feature is that it does not require specific technological materials. It can also work with natural elements such as plants, leaves, soil, fruit, or simply the human body. [2] Sensors, digital models, or nature-based interface platforms can therefore amplify existing signals, without replacing them.
We have become an indoor generation. We spend most of our lives inside buildings, on public transportation, and in enclosed spaces. Home, school, office, gym, and shops: we rarely spend time outdoors, even though we often think otherwise. Recent studies show that people believe they spend about 60–70% of their time indoors, but the reality is very different: on average, we spend up to 90% of our day indoors. This change has occurred in a very short time compared to the history of human evolution, which has seen us live outdoors for hundreds of thousands of years, following the natural cycles of light and dark. This distance from nature affects the body, the mind, and the way we relate to the environment. In this context, caring for plants becomes a simple yet meaningful gesture, especially for certain generations more sensitive to these issues.
The Physical and Mental Effects of an Indoor Lifestyle
Living primarily indoors has concrete consequences. Indoor air is often more polluted than outdoor air, even in cities. Building materials, furniture, cleaning products, and simple daily activities like cooking or breathing increase levels of CO₂ and harmful substances. The lack of natural light also has a significant impact. Our bodies use daylight to regulate our sleep-wake cycles. Spending little time outdoors can cause sleep problems, fatigue, difficulty concentrating, and mood swings. In many cases, it also contributes to stress, anxiety, and seasonal depression. Despite this, we’re often unaware of how little contact we have with nature. This creates a gap between perception and reality that makes it difficult to change habits.
Image 2. Graph showing average time spent indoors over 24 hours – from Bauenmitholz
Why Millennials and Gen Z Are More Drawn to Plant Care
Image 3. Graph with the different generations divided by age group- from Fineco Bank
Research shows that millennials and Gen Z are currently the generations most involved in caring for indoor plants. This doesn’t mean they have more plants than other generations, but they purchase and care for them more frequently and with greater attention. There are several reasons:
They more often live in apartments without gardens
They have a strong connection to mental well-being
They are more sensitive to environmental issues
They use plants as a form of self-care For many young adults, caring for a plant isn’t just a hobby, but a way to slow down, take responsibility, and reconnect with something alive. It’s no coincidence that a large percentage of millennials say that plants make them happier and more optimistic about the future.
Indoor Plants as a Response to Disconnection from Nature
Indoor plants can mitigate some typical problems of indoor living:
They improve the perception of air quality
They introduce natural variations into the space
They make the passage of time visible
They promote routine and attention Even when the biological impact is limited, the psychological effect is strong. A plant changes, grows, and reacts. It’s the opposite of a screen that’s always the same. This opens up an interesting space for interaction design.
Nature-Based Interaction Design: From Screens to Living Interfaces
We are the indoor generation, but that doesn’t mean we have to give up contact with nature. Younger generations, accustomed to complex digital interfaces, seem to increasingly appreciate simple, natural interactions. Plants offer just that: interaction based on observation, time, and slow feedback. For interaction design, this means rethinking the role of interfaces:
fewer screens
more living objects
more relationships, less control Nature-based user interfaces can help rebuild a connection with the environment, especially for those who mostly live indoors.
[2] A. Author et al., “Article title,” International Journal of Environmental Research and Public Health, vol. 19, no. 12, Art. no. 7454, 2022. [Online]. Available: https://www.mdpi.com/1660-4601/19/12/7454
[5] P. Ahmmadi, M. Rahimian, and R. G. Movahed, “Theory of planned behavior to predict consumer behavior in using products irrigated with purified wastewater in Iran,” Journal of Cleaner Production, vol. 296, 2021, Art. no. 126359, doi: 10.1016/j.jclepro.2021.126359.
[6] P. T. Thuy, N. T. Hue, and L. Q. Dat, “Households’ willingness-to-pay for mangrove environmental services: Evidence from Phu Long, Northeast Vietnam,” Trees, Forests and People, vol. 15, Art. no. 100474, 2024, doi: 10.1016/j.tfp.2023.100474.
[7] C. J. Armitage and M. Conner, “Efficacy of the theory of planned behaviour: A meta-analytic review,” British Journal of Social Psychology, vol. 49, no. 3, pp. 471–499, 2010.
[8] X. Liu, H. Lindhjem, K. Grimsrud, E. Leknes, and E. Tvinnereim, “Is there a generational shift in preferences for forest carbon sequestration versus preservation of agricultural landscapes?” Climatic Change, vol. 176, no. 9, 2023, doi: 10.1007/s10584-023-03588-z.
[9] B. Satinover, J. Nichols, and J. W. Holt, “A comparison of sustainability attitudes and intentions across generations and gender: A perspective from U.S. consumers,” Cuadernos de Gestión, vol. 23, no. 1, pp. 51–62, 2023, doi: 10.5295/cdg.211647bs.
[10] X. Li, J. Dai, X. Zhu, J. Li, J. He, Y. Huang, X. Liu, and Q. Shen, “Mechanism of attitude, subjective norms, and perceived behavioral control influence the green development behavior of construction enterprises,” Humanities and Social Sciences Communications, vol. 10, no. 1, p. 266, 2023.
[11] W. J. Fisk, “Health and productivity gains from better indoor environments and their relationship with building energy efficiency,” Annual Review of Energy and the Environment, vol. 25, pp. 537–566, 2000. [Online]. Available: https://www.annualreviews.org/doi/pdf/10.1146/annurev.energy.25.1.537
