For a long time, using a computer was a merely cognitive thing to do – a dialogue. Those were the times in which computers were sitting at desks, and were merely operated through buttons and mice, and displayed information either as text or graphics. This paradigm has already fallen, now that computers are ubiquitous in our everyday lives, and other paradigms are perhaps soon to follow.
Recent developments in the research field of Human-Computer Interaction point to emerging styles of interaction that make use of our very abilities as human beings, putting us directly in touch with the digital world.
If we look at the ways in which we interact with computers across the last three decades, a number of major changes become evident: In the 1980’s, we have interacted with computers in textual ways. In the 1990’s, we have changed to graphical interaction – and in the most recent decade, again, these ways are changing.
One theory, as proposed by Paul Dourish in 2001 in his book ‘Where the Action is’, to conceptualize these recent changes, is ‘Embodied Interaction’. Dourish points out that computing is moving into the social and the physical space, and he proposed the term ‘Embodied Interaction’. Advantageously, these new styles of computing draw upon skills that we already have, skills that we, as human beings, embody.
In this article, we will look at three series of prototypes that illustrate what ‘Embodied Interaction’ is – in the different physical and social spaces that we live in.
How can we make digital content graspable?
Nowadays, our lives take place in two worlds: on the one side in the digital world, and, on the other side, in the physical world. While many things happen in the digital world, and while it is in many ways an influential place, things in the digital world are not tangible for us. Many people do not understand the digital – it is hard to grasp.
As humans, we are very skilled in engaging with the physical world. So the question is: How can we use our everyday, real-world skills to interact with digital contents? How can we move things from the digital world into the physical world? If we look at emerging interaction techniques, it is obvious that this is already the trend: the iPhone’s touch and the Wii’s bodily activity make clear that interacting with the digital is getting more and more physical. The question is: what’s next?
In our research, we have developed a series of three prototypes that provide a tangible glimpse on how the digital could be made graspable.
The first prototype involves weight as a representation for digital content. It is a mobile phone-shaped box that, on its inside, features a motorized weight. This weight can be moved, and thereby the device’s center of gravity can be moved. This allows for a variety of applications:
Firstly, the device’s ergonomics can be changed. If it would usually fall out of the user’s hand, the center of gravity can be moved to balance the device automatically. Secondly, content on the device’s inside can be made feelable – especially the distribution of content ‘in the device’ can be represented, e.g. if the majority of content is on the device’s right side (for instance, in a list of songs), the device would also be heavier on the right side. Thirdly, a shifting weight can be used to represent contents that are external to the device, in a certain direction. This could be useful for navigation, in which the device would physically point, by shifting its center of gravity, into a direction that would remain the same when the user turns: as a haptic compass.
The second prototype draws upon shape change as a means of displaying digital contents. It is equipped with a set of motors that allow for the actuation of the device’s geometry. In doing so, they allow for making the device thin in the pocket, and thick, ergonomically shaped when held in hand. Furthermore, they allow for the physical display of content amount or direction, similar to the previously described weight-based variant. More content can simply be thicker, as in an e-book that has, when read from the beginning, all of its pages on the right side (resulting in a feelable thickness on that side), slowly moving over to the left side while progressing through the book. Also directional information can be conveyed, shifting the device’s thickest point into the desired direction.
The third prototype of this series draws on our human ability of empathy – as social beings, we are well able to feel how other people or beings are. Inspired by this ability, the ‘living mobile’ provides information about missed calls or text messages through breath and pulse. The device vibrates in a heartbeat-like manner, and has a motorized chest that moves in a breathing-like movement. In cased of no missed calls or events, the phone will behave calmly, while it will utter excitement in its movement if it needs the user’s attention.
How can we make mobile phone calls more polite?
The blurring of the digital and the physical can be observed not only in interacting with content – it is also visible when it comes to interacting with other people through digital channels. Here, these channels may interfere with the ways we normally communicate, and thereby result in impoliteness.
A particular social problem of mobile phones is the issue of incoming calls in busy situations. Even though users are able to tell who is calling, they generally do now know whether the matter at hand is important or not. Similarly, users may find it inappropriate to call just for a chat, as they are unable to express that their call is not important.
Users often find themselves in a conflict when noticing an incoming call – should they interrupt what they are doing, and take the call? Or should the reject the call? The latter is often considered impolite, given the fact that the caller not even had a chance to express what the call is about.
To overcome this issue, we have built a prototype that employs a pressure-sensitive dial button. A more important call can be placed by pressing the button stronger, while also gentle calls can be placed – by pressing the dial button only gently. The prototype also has a filter, allowing to change vibration, ringing and also voicemail behaviour depending on the urgency of the incoming call.
How can we make phone calls more emotional?
Giving users a feeling for digital content in their device shows that there is great potential in making the digital physical. But in mobile phones, the even more relevant field of research could be another one: giving two users a feeling for each other.
In their current form, mobile phones are well-suited for one of the major reasons of telecommunication: to exchange information. For the other major reason of telecommunication – the whish for nearness – speech, video and text might be not all we can do: sometimes, we just want to be in touch. We developed a series of prototypes that explore this field.
The first prototype, the ‘grasping mobile’, telecommunicates pressure and potentially allows for an experience of holding hands over a distance. It is equipped with a strap on its backside, into which the user’s hand is placed. The pressure exerted by the telecommunication partner’s hand on their phone is then telecommunicated through a motor that pulls the strap tighter, and vice versa.
The second prototype, the ‘kissing phone’ telecommunicates moisture. While there are many people one may not want any kind of ‘moisture-enabled telecommunication’ with, there may be some exceptions worthwhile exploring. The technology used in the prototype is a semi-permeable membrane that lets liquids out, but not in, and a motorized sponge that is wettened and pressed against the membrane.
The third prototype, the ‘whispering phone’, telecommunicates airstream. In normal speech, airstream is generally involved, but only felt in close-by conversations, like whispering or sighing. The prototype involves a set o air jets on the phone’s surface, allowing for different styles of airstreams.
These prototypes render a certain future vision of telecommunication tangible – and in doing so, they provoke questions: how much nearness do we want? What kinds of privacy protection will we find ourselves needing in the future? They also demonstrate the value of prototyping – in making a future vision experienceable today they provide concreteness to an otherwisely abstract thought, and thereby allow for discussion.
A new world of interactions
The works described here are only a small section of the current developments in interaction design. What is obvious, though, is that the new omnipresence of computation brings along a whole new world of interactivity. The ways in which we manipulate and experience the digital have undergone radical changes in the last decades, and it will be exciting to see how these ways will change in the future. In the end, it’s probably not humans that should get more technical.
It’s technology that should get more human.
Fabian Hemmert will be one of the keynote speakers at Interaction 12. It is the fifth annual conference hosted by the Interaction Design Association (IxDA). Each year, IxDA aims to gather the interaction design community to connect, educate, and inspire each other. This year it is held in Dublin, Ireland.