Multisensory Design and the use of Metaphors in User Experience

Image for post
Image for post

When I started my postgraduate degree in Human-Computer Interaction & Design I did not have any previous formal education nor training in design thinking. With my background in IT, I was used to think of product design in terms of functionality. What is the task of the product? What is the user’s purpose when using the product? These are all valid questions that designers and engineers alike need to take into account when creating pretty much anything for anyone. My largest surprise when I took my very first design course that was not directly related to software engineering — a class named “multisensory design” from the University of Twente in the Netherlands — was discovering that these questions are barely enough when it comes to design thinking, and rarely enough when it comes to good design thinking.

Beyond multimodality

I chose multimodal interaction as the specialisation route for my study track because that something that I was familiar with: I wrote my bachelor thesis on the topic of combined visual and auditory stimuli in multimodal user interfaces. Most user interfaces today are at the very least bimodal: combining two or more of visual, auditory, and haptic/tactile stimuli to allow for the display and access of information. In software design, this means having to deal with lots of graphic elements, earcons (auditory icons), touchscreens, keyboards, computer mouses or other types of input. Again, the focus is to make it easy for the user to perform the required task. Because of this, I felt in-between excitement and confusion when I found out that our assignment for the multisensory design course did not involve any user problem nor task, no requirements gathering, no user interviews, technically not even a product nor a service. The assignment instead asked us to design a “ritual” that could assist someone in the self-regulation of positive feelings. We had to design for mood in order to facilitate resilience — one’s ability to recover from negative experiences and focus on the positive aspects of life. Can design thinking really be that powerful?

Metaphors and the interconnections of the senses

One of the suggested readings for our course was Sensation — The New Science of Physical Interaction by Thalma Lobel. While our teacher warned us that the book was not really good for learning design practices (and with reason), it is to date one of my favourite reads. The author has a very personal way to show how every minor thing surrounding us has some sort of weight on our perception. I purposefully used the word weight to which she dedicates a whole chapter, titled “Don’t Take This Lightly: The Importance of Weight” in which she enlists several figures of speech for weight: “weighting’s one options”, “carrying a lot of weight”, “weighting down”, “the weight of the world”. The importance of weight is that weight is a synonym for something important, and this affects our way to look at things that are (or appear to be) physically heavy. It’s the main reason why many ancient buildings and monuments were built with heavy materials and huge in size — think of the pyramids or the great wall of China. I went on a trip to Budapest, not long ago, and a tour guide explained how the Cathedral and Parliament were built to be exactly the same height to signify that the church and the government had equal power in Hungary and hence held the same importance. In UX/UI design, the most blatant example of this is the use of bold formatting for characters, which makes certain words appear heavier and hence more important than others.

Not unlikely the weight, temperature also affects our perception: participants of an experiment were asked to describe a fictious person after holding a cup of coffee in an elevator. Those who held a warm cup of coffee were more likely to describe this person as generous, good-natured, sociable and caring, while those who held a cold cup of coffee picked the adjectives ungenerous, irritable, antisocial and selfish more often. In Europe, there is a common stereotype that people from Mediterranean countries (Italy, Spain, Greece) have “warm” personalities, being generally friendly and sociable; while people from northern countries (Germany, Scandinavia, Russia) are more “cold”. Interestingly, there are rules of thumb in web design which seem to favour cold colours to warm ones in order to provide a sense of professionalism while warm colours may be seen as childish.

Another important metaphor that one needs to take into account when designing is that of space. This is a concept that is of particular relevance for those interested in Digital Anthropology, as the concepts of time and space have changed quickly and significantly due to the advances of modern technology and telecommunication. Cultural anthropologist Edward T. Hall defined space in terms of “invisible boundaries” such as those of territoriality, personal space, and multisensory space. In this regard, people from different cultures may have different expectations regarding whether it is socially acceptable to smile, touch, or talk to another person. In Italy, talking loudly in public is common and widely accepted, people are somewhat laid back when it comes to hugging and cheek-kissing friends and acquaintances is common in many regions in the south. In the UK, making eye contact with strangers is often a big no-no and being in the proximity of another person, almost bumping into them but without any physical contact may be enough for one of the parties involved to be expected to apologise. Physical space may also reflect the emotional distance between two parties, or their relative position of power. Generally, higher positions of power are associated with the “top” while less powerful positions occupy lower vertical spaces. Many social media spaces often decide to highlight “top” stories, or to stack recommended content on top of the page in order to give it priority, in order to keep the user engaged. Whenever I rearrange apps and widgets on my phone, I make sure that the clock, weather forecast, and messaging apps I use most often are always on top of the screen, so that I can access them instantly as soon as I turn my phone on.

Conclusion

If you are interest in the topic, I suggest you give Sensation a read — it goes further into detail on how sensory metaphors affect our perception and also deals with the senses of taste and smell. Multisensory design is a powerful tool that allows experience design to go beyond mere usability. Whether you are designing a product, interface, or service chances are your end user will have to engage some or all of their senses and it is your responsibility as a designer to ensure that each of them blends into an overall positive user experience.

Sensation: The New Science of Physical Intelligence by Thalma Lobel

Image for post
Image for post

By the world’s leading expert on the psychology of physical intelligence comes this exciting, new view of human behavior that explains how the body profoundly and unconsciously affects our everyday decisions and choices, and will appeal to readers of Predictably Irrational and Emotional Intelligence.

From colors and temperatures to heavy objects and tall people, a whole symphony of external stimuli exerts a constant influence on the way your mind works. Yet these effects have been hidden from you — until now. Drawing on her own work as well as from research across the globe, Dr. Thalma Lobel reveals how shockingly susceptible we are to sensory input from the world around us.

Thank you for reading! If you enjoy this topic, please join in the discussion in the comments below! You may also show appreciation by leaving a few claps tapping the👏 button or by sharing this article with your network.

UX Researcher @ FlixBus. Pan-European, uprooted globetrotter. Idiosyncratic. Passionate about UX, Human-Computer Interaction, and Digital Anthropology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store