She couldn’t do those things she loved anymore. Horseback riding was physically impossible, just like playing the piano. Those late night dinner parties were just too intensive. That hurt Maria, but what hurt her most was that her friends sometimes made comments that she looked so well. Her friends thought that Maria was winning the struggle against her illness, while in fact Maria’s treatment was not catching on. She was slowly dying.

While I did research on the context of struggling with a severe illness I learned that we lose one fourth of our social environment when we are diagnosed with a severe illness. That is quite a lot, especially in the time that you need that social contact the most. It creates support, self esteem and makes the illness more bearable.

Quite frankly I was stunned by this. Because I have a technical background, I know there are many amazing technologies which change the world rapidly, especially in the way we communicate. But yet we cannot utilize these technologies to communicate about our thoughts, feelings and health.

The Necklace

It was after that experience I started to design for ‘intimate communication’. The Necklace became the first design in this genre. It is a piece of jewelry for breast cancer patients. After each important moment in your struggle against this illness you add a new link. By gently pushing a link of the necklace onto your skin it withdraws a small blood sample without any feeling of discomfort. This link then changes color based on your blood values. The deeper the color of the link the healthier you are. Such information infused decoration allows you to see, at any time of the day, how your recovery is progressing and lets you communicate it with your love ones whom you learned how to read The Necklace. In this way this unique piece of jewelry symbolizes your personal story.

Design for Debate

The interesting thing is that The Necklace is not an actual product, but a tangible and interactive future scenario about that our intimate communication could be like within the next ten years. This design is currently on tour with the Nano Supermarket, a traveling exhibition full of speculative products which could be realized within the next ten years with the help of nanotechnology. During this tour I got a lot of mixed comments; either people loved the design, or deeply hated it. They explained how it could have helped them in their disease, or how they would hate to give up their privacy.

Because it is a tangible, interactive and realistic design people can engage and experience it. They can actually talk about it how it would affect their lives, because it is there, right in front of them. This was exactly the point of the design. It was not designed to be a future product, but to be debated about. Through these comments I learned a lot on how people experience communication in the context of health and on how they see products influencing this communication. It is a design for debate.

User Experience

I believe that we need such realistically crafted future scenarios in order to investigate what we want our future to be like; specially in the case of ‘intimate communication’ via technology. We can than start to create new dimensions in the way we communicate via technology and become closer. The user experience of these scenarios is therefore very important. They either make or break the illusion, and therefore the discussion. This experience is needed in Design for Debate.

As Sherry Turkle, Professor of the Social Studies of Science and Technology at the Massachusetts Institute of Technology, says: “Objects are carriers of experiences and emotions”. Designers can and should create those objects. The people who engage with the objects then are able to create the future.

The project was announced in Google style, by placing a short post and video on Google+: “A group of us from Google[x] started Project Glass to build this kind of technology, one that helps you explore and share your world, putting you back in the moment. We’re sharing this information now because we want to start a conversation and learn from your valuable input.”

Taking steps

It’s interesting to see that a technology driven company like Google is capable of moving augmented reality goggles beyond ugly machines on your nose. They’ve managed to create appealing devices that look a like jewelry. The New York Times states that “Project Glass could hypothetically become Project Contact Lens. Mr. Parviz, who is also an associate professor at the University of Washington [...] recently built a tiny contact lens that has embedded electronics and can display pixels to a person’s eye.”

Service

The video shows how the device is integrated into the tools Google offers us, ranging from Gmail to Google Chat and Google+. Of course this is a logical step, but it’s also the opportunity for Google to once again become a game changer. This time it’s not a battle with Yahoo! and Altavista, but a direct assault on Apple. Google started with their own phone and OS, and are now moving further into the hardware world. They’ve understood that the battle isn’t fought and won over just a lot of software users, but about the importance of delivering a high quality service on all touchpoints.

Google is open, Apple not

And Google has one other benefit: they are open. In contrast to Apple they are launching a lot of products and services and are constantly testing them. They are very open to the community and want to get their thoughts and insights. See how they use their own products to get in touch with us. And it’s not a corporate voice, but it’s the actual people in the design teams that are interested in connecting with us. Google wants to start a conversation, where Apple has taken the position of a messiah. And that for me is the most interesting thing to observe when I see Project Glass getting announced.

His blog post shows him map out various interactions, and their resulting spaces:

House Study

A new type of space from the diagram: Maker Space

I don’t necessarily agree with all of his points (he believes that web apps are primarily linear), and also agree with one commenter pointing out that he didn’t have to use a webapp as the concepts of flows in architecture have been around for decades. Still, it’s an interesting concept that puts the A back into IA.

I also noticed one of his points in his rationale.

the web is not a graphic design medium but a product design one.

The correlation between web and product seems to be becoming noticed as of late (for example Jeff Croft’s ‘In 2012, Let’s Stop Talking Web Design and Start Talking Product Design’ or Smashing Magazine’s ‘What Successful Products Teach Us About Product Design’).

Above all, it’s a reminder at how we can not only draw inspiration from other disciplines, but also feed it back the other way.

There are still parameters to input and many options to choose before we can finally hit the ‘go’ button on many devices. Take a digital camera for instance. Advanced technology must be able to help a camera tell which scene mode the camera should choose. A toaster must know how long it will take to have nicely toasted bread. Recent gadgets are now able to tell where they are, who is there, what they are facing and many other things, so they should be able to do more. It’s like a very intelligent person who is blindfolded wearing earplugs.

I feel that way especially when I put some food to cook in a microwave oven. I know what the food should be like for me to eat. I know it shouldn’t be burned. I know it should be defrosted. But I’m not a cook and don’t know exactly how long it should be cooked. And that’s why I thought of this concept design.

WYSIWYG Microwave Oven Concept

Let us adjust the ‘how-we-want-it’ setting and skip all the details. Or rather, the ‘what-it-should-be-like’ setting because visual aid helps us picture the results much more easily than verbal expressions do.

Put a dish in it and this oven tells what kinds of foods are in it.

All you need to do is to adjust the 'look' of the food displayed on the glass by dragging the button. You can control the cooking time while looking at what it will be like after cooking the adjusted length of time. You'll know exactly when it boils or burns.

By the way, I wish a microwave oven could be movable so that it could get close to the table when necessary.

This post by John Tolva is heralded as “critical reading if you have any interest whatsoever in networked cities and citizenries” by city-ubicomp expert Adam Greenfield.

As Chief Technology Officer for the City of Chicago, he uses examples from city initiatives (Bus Tracker, Train Tracker and potential other civic startups) to propose the future of urban cities:

… [think] of the city itself as an open platform with an API. Physical objects generate data that can be combined, built upon, and openly shared just as it can be from the data portal. The difference in this scenario is location. Where much of the data in the portal is geo-tagged, data coming from the built environment would be geo-actionable. That is, in the city-as-platform scenario certain data is only useful in the context of the moment and the place it is accessed.

Just don’t call it an ‘urban OS’:

… I’m growing skeptical of calling all this an operating system, at least in the sense we traditionally do. Much of the talk of an urban OS focuses solely on centralized control. But if you’re true to the analogy of a computer operating system it would have to be a platform for others to build applications upon. In truth, this is a lot more like a robustly deployed, well-documented set of fault-tolerant API endpoints than it is an OS.

There’s been a lot of interest in urban spaces in the last few years from people such as Adam Greenfield,  Andrea Resmini and Luca Rosati (who we also covered in this year’s EuroIA conference notes), and Dan Hill (who we also covered at Interaction 10). However, Tolva’s perspective is particularly heartening given that he represents the public sector, and represents a growing number of authorities realising the future role of open data.

—

City of Chicago image NC-BY-CC by accentstage

For a long time, using a computer was a merely cognitive thing to do – a dialogue. Those were the times in which computers were sitting at desks, and were merely operated through buttons and mice, and displayed information either as text or graphics. This paradigm has already fallen, now that computers are ubiquitous in our everyday lives, and other paradigms are perhaps soon to follow.

Recent developments in the research field of Human-Computer Interaction point to emerging styles of interaction that make use of our very abilities as human beings, putting us directly in touch with the digital world.

Embodied interaction

If we look at the ways in which we interact with computers across the last three decades, a number of major changes become evident: In the 1980’s, we have interacted with computers in textual ways. In the 1990’s, we have changed to graphical interaction – and in the most recent decade, again, these ways are changing.

One theory, as proposed by Paul Dourish in 2001 in his book ‘Where the Action is’, to conceptualize these recent changes, is ‘Embodied Interaction’. Dourish points out that computing is moving into the social and the physical space, and he proposed the term ‘Embodied Interaction’. Advantageously, these new styles of computing draw upon skills that we already have, skills that we, as human beings, embody.

In this article, we will look at three series of prototypes that illustrate what ‘Embodied Interaction’ is – in the different physical and social spaces that we live in.

How can we make digital content graspable?

Nowadays, our lives take place in two worlds: on the one side in the digital world, and, on the other side, in the physical world. While many things happen in the digital world, and while it is in many ways an influential place, things in the digital world are not tangible for us. Many people do not understand the digital – it is hard to grasp.

As humans, we are very skilled in engaging with the physical world. So the question is: How can we use our everyday, real-world skills to interact with digital contents? How can we move things from the digital world into the physical world? If we look at emerging interaction techniques, it is obvious that this is already the trend: the iPhone’s touch and the Wii’s bodily activity make clear that interacting with the digital is getting more and more physical. The question is: what’s next?

In our research, we have developed a series of three prototypes that provide a tangible glimpse on how the digital could be made graspable.

The first prototype involves weight as a representation for digital content. It is a mobile phone-shaped box that, on its inside, features a motorized weight. This weight can be moved, and thereby the device’s center of gravity can be moved. This allows for a variety of applications:

Firstly, the device’s ergonomics can be changed. If it would usually fall out of the user’s hand, the center of gravity can be moved to balance the device automatically. Secondly, content on the device’s inside can be made feelable – especially the distribution of content ‘in the device’ can be represented, e.g. if the majority of content is on the device’s right side (for instance, in a list of songs), the device would also be heavier on the right side. Thirdly, a shifting weight can be used to represent contents that are external to the device, in a certain direction. This could be useful for navigation, in which the device would physically point, by shifting its center of gravity, into a direction that would remain the same when the user turns: as a haptic compass.

The second prototype draws upon shape change as a means of displaying digital contents. It is equipped with a set of motors that allow for the actuation of the device’s geometry. In doing so, they allow for making the device thin in the pocket, and thick, ergonomically shaped when held in hand. Furthermore, they allow for the physical display of content amount or direction, similar to the previously described weight-based variant. More content can simply be thicker, as in an e-book that has, when read from the beginning, all of its pages on the right side (resulting in a feelable thickness on that side), slowly moving over to the left side while progressing through the book. Also directional information can be conveyed, shifting the device’s thickest point into the desired direction.

The third prototype of this series draws on our human ability of empathy – as social beings, we are well able to feel how other people or beings are. Inspired by this ability, the ‘living mobile’ provides information about missed calls or text messages through breath and pulse. The device vibrates in a heartbeat-like manner, and has a motorized chest that moves in a breathing-like movement. In cased of no missed calls or events, the phone will behave calmly, while it will utter excitement in its movement if it needs the user’s attention.

How can we make mobile phone calls more polite?

The blurring of the digital and the physical can be observed not only in interacting with content – it is also visible when it comes to interacting with other people through digital channels. Here, these channels may interfere with the ways we normally communicate, and thereby result in impoliteness.

A particular social problem of mobile phones is the issue of incoming calls in busy situations. Even though users are able to tell who is calling, they generally do now know whether the matter at hand is important or not. Similarly, users may find it inappropriate to call just for a chat, as they are unable to express that their call is not important.

Users often find themselves in a conflict when noticing an incoming call – should they interrupt what they are doing, and take the call? Or should the reject the call? The latter is often considered impolite, given the fact that the caller not even had a chance to express what the call is about.

To overcome this issue, we have built a prototype that employs a pressure-sensitive dial button. A more important call can be placed by pressing the button stronger, while also gentle calls can be placed – by pressing the dial button only gently. The prototype also has a filter, allowing to change vibration, ringing and also voicemail behaviour depending on the urgency of the incoming call.

How can we make phone calls more emotional?

Giving users a feeling for digital content in their device shows that there is great potential in making the digital physical. But in mobile phones, the even more relevant field of research could be another one: giving two users a feeling for each other.

In their current form, mobile phones are well-suited for one of the major reasons of telecommunication: to exchange information. For the other major reason of telecommunication – the whish for nearness – speech, video and text might be not all we can do: sometimes, we just want to be in touch. We developed a series of prototypes that explore this field.

The first prototype, the ‘grasping mobile’, telecommunicates pressure and potentially allows for an experience of holding hands over a distance. It is equipped with a strap on its backside, into which the user’s hand is placed. The pressure exerted by the telecommunication partner’s hand on their phone is then telecommunicated through a motor that pulls the strap tighter, and vice versa.

The second prototype, the ‘kissing phone’ telecommunicates moisture. While there are many people one may not want any kind of ‘moisture-enabled telecommunication’ with, there may be some exceptions worthwhile exploring. The technology used in the prototype is a semi-permeable membrane that lets liquids out, but not in, and a motorized sponge that is wettened and pressed against the membrane.

The third prototype, the ‘whispering phone’, telecommunicates airstream. In normal speech, airstream is generally involved, but only felt in close-by conversations, like whispering or sighing. The prototype involves a set o air jets on the phone’s surface, allowing for different styles of airstreams.

These prototypes render a certain future vision of telecommunication tangible – and in doing so, they provoke questions: how much nearness do we want? What kinds of privacy protection will we find ourselves needing in the future? They also demonstrate the value of prototyping – in making a future vision experienceable today they provide concreteness to an otherwisely abstract thought, and thereby allow for discussion.

A new world of interactions

The works described here are only a small section of the current developments in interaction design. What is obvious, though, is that the new omnipresence of computation brings along a whole new world of interactivity. The ways in which we manipulate and experience the digital have undergone radical changes in the last decades, and it will be exciting to see how these ways will change in the future. In the end, it’s probably not humans that should get more technical.

It’s technology that should get more human.

Interaction 12

Fabian Hemmert will be one of the keynote speakers at Interaction 12. It is the fifth annual conference hosted by the Interaction Design Association (IxDA). Each year, IxDA aims to gather the interaction design community to connect, educate, and inspire each other. This year it is held in Dublin, Ireland.

A nice one for anyone interested in urbanism. Via Vicky Teinaki.

Duration: just under an hour

Interaction and experience designers are often only afforded a very focused, tactical design problem to solve, but the real impact comes when you can widen that lens and solve larger design issues. But how can a designer gain this increased breadth of perspective and scope? In this article, I’ll explain how we turned a tactical hospital signage study into an exploration of the broader service experience for its visitors.

To a great mentor, this article in memorium to Sylvia Harris.

It’s not always instinctual to consider a hospital as a business, making decisions every day to manage against a financial bottom line. Perhaps we want to forget that hospitals have other interests they are balancing beyond our own care. Regardless, the business implications for a poor service experience are no different at a hospital than with any other client; in fact, they are elevated given what’s at stake: people’s health. I had the opportunity to work with a large hospital in the Northeast US on a pro bono visitor experience study. The hospital was ranked in the top 10 of US hospitals for their clinical care, but the patient services group felt that the experience of navigating the hospital – or wayfinding – didn’t match the excellence in care. The poor visitor experience was leading to late or missed appointments, which has a tremendous effect on scheduling and budget, not to mention leading to frustrated visitors, and the team wanted to address these issues through an improved wayfinding system.

Expanding Our Lens @ the Onset

We collaborated with Sylvia Harris Research & Design and Two Twelve Associates, who were in the process of redesigning the hospital branding, on the study. The focus of our work was intended to be on the tangible wayfinding system and signage. However, while working with the patient services stakeholders to solidify the project objectives, we asked them questions beyond just the wayfinding system:

• What resources do visitors use to prepare for their visit before they even arrive on campus (e.g. web site, campus maps, calling ahead)? What can improve the pre-visit preparation?
• What is the process for appointment scheduling? Who’s involved?
• Who are all of the constituents involved in the hospital service: patients, loved ones, volunteers, nurse staff, physicians, etc.? What is their role and influence in the service?
• Beyond the physical structure of the building and the signage, what else do visitors rely on to navigate the hospital?
• What are the diversity of reasons and scenarios for why someone comes to the hospital?

The exploration enlightened the stakeholders to the many facets required to create a successful visitor experience beyond solely the wayfinding system itself. We then co-created the long-term goal of the project, which you’ll notice doesn’t mention signs or wayfinding: to reduce patient & visitor stress and lateness, to ensure positive first and last impressions between the hospital and patients & visitors, and ultimately to match the excellence in care with excellence in experience.

Garnering Holistic Insights

Key sign indicating 'waiting room' is located to left of door, but not where most visitors approach (from right). Visitors walked straight past the room and asked clinical staff for directions.

Once we established the broader service lens through which we’d be evaluating the visitor experience, we needed to create a study that was equally holistic but manageable because it was a pro bono project.

To understand the interactions between the various hospital service elements, we observed for several hours the information exchange between patients & visitors and the information desk, which frequently represented the start of the on-site hospital experience. We noted the following: materials that the visitor had prepared (e.g., appointment slips were common), the request for help, clarity of directions given, comprehension by the visitor, and their general mental state. We then shadowed a sub-set of them, approximately 20, en route to their destination, noting how well they followed instructions, how they used the space and signage, and if they relied on any other resources along the way.

Doing so provided insights and recommendations beyond just the wayfinding system. For example, visitors consider all humans who work in the hospital – from security guards to surgeons – to represent the service provider, equally responsible to help them find their way. Therefore, hospital training for communicating with visitors should extend beyond just the info desk personnel. Also, the emotional aspects of being in a hospital cannot be underestimated. We frequently witnessed visitors enter a spiral of confusion about where they were, which led to them being overwhelmed and frantic, which in turn led to more confusion about the experience; the design solution – from the signage to the people – needs to not only empathize with this mindset, but help alleviate it.

visitors consider all humans who work in the hospital – from security guards to surgeons – to represent the service provider

The uninviting industrial door and fire exit (at the end of the hall) intuitively make visitors question whether they are going the right way. Signage earlier in the experience (front door) went unnoticed because of its location.

Surveys and interviews in a few waiting rooms helped us triangulate what we were observing through the shadowing exercise with the visitors’ own reflection on the experience without adding unnecessary stress or distraction to their already complex experience. While conducting these 50 interviews, we observed hundreds of interactions between the office staff and visitors. We observed confusing conversations over the phone about where they needed to go for an appointment or people showing up for their appointment, only to be told they needed to go somewhere else. No matter how intuitive the wayfinding system is, if the broader service experience and interactions among the clinical staff, the support staff, the information desk and the visitors is not orchestrated effectively, the system will continue to falter.

No matter how intuitive the wayfinding system is, if the broader service experience and interactions among the clinical staff, the support staff, the information desk and the visitors is not orchestrated effectively, the system will continue to falter.

Our last method was conducting scenario walk-throughs with six stakeholders and volunteers, giving them “tasks” (e.g., “find the Labor and Delivery room”) while we shadowed them. We asked them to articulate their behaviors and thoughts and probed with questions along the journey, which is not dissimilar from conducting a usability test. Involving stakeholders directly in the process illustrated to them first-hand what it feels like to be in a visitors’ shoes, interacting with the space, the signage, the people, and other various service elements of the hospital. Doing so also helped ensure that the stakeholders would continue to support the broader service lens we were applying to the project vs. suggesting we simply “evaluate the signage.”

Lessons for Designers

The hospital project originally intended to focus on the tangible aspects of the wayfinding system, but we very quickly acknowledged the need to explore the broader service experience of the hospital. Doing so requires a different perspective than we are accustomed to, one which does not have visitors as the ‘center’ of that experience and one which doesn’t focus on tangible design elements. Rather, it’s a decentralized, orchestrated system of elements with no single entity as the focus and one where intangible aspects – the communications between staff-visitors, the emotional considerations of the hospital context, the time it takes to find one’s way – are as important as the tangible ones.

A debate currently exists regarding service design and just how different it is from experience design. In short, if experience design is the proactive and strategic creation of a structure across interactions, channels, people, technologies, processes, objects, etc. within which you hope people have a certain type of experience, then I agree that service design has significant overlap with experience design. However, I fear that XD has become just another evolutionary definition of “usability” and “user experience” and question whether we’ve been able to successfully put the definition into practice.

A service by definition is a co-created value exchange among constituents, and the experience of that service succeeds or fails based on how well-orchestrated all of the service elements are. The burden and responsibility to design it right is deep. For experience design and interaction design professionals looking to have more impact in the design world, service design naturally affords us that opportunity because of its implicit broad and holistic perspective. Considering design problems previously thought of as tangible, focused, product-oriented experiences instead as decentralized, multi-faceted, service experiences may actually allow us the impact we aspire to have.

Here’s a beautifully crafted video from Urbanscale and Nordkapp, which shows how the many urban screens dotted around Helsinki could be used for more than just advertising and basic wayfinding, but as part of a wider “operating system for cities”.

Duration: five minutes

Matt Jones and Jack Schulze (BERG), talking about the often intangible materials that we as interaction designers work with. (From the SVA MFA in Interaction Design Spring Lecture Series.)

Duration: just over an hour (but it feels like much less).