I read Marshall McLuhan years ago when I was knee deep in media theory and cognitive psychology, but it took a trip to the IA Summit in Memphis last week to bring that back – and I immediately began thinking about the materials, medium and methods we choose to use in our ideation and exploration phases and how it impacts our designs and solutions.

Here’s Jesse James Garrett at the recent IA Summit 2009 in Memphis quoting Michael Wesch quoting Marshall McLuhan:

We shape our tools, and then our tools shape us.
~ Marshall McLuhan

I think that design materials, methods and problems are all tied together in a mutual dependency. Consider the example of an online training for an interactive system. The material is set to be HTML,  and perhaps Adobe Flex. This controls what the designers can do and how they perceive the problem. There are some things the designers cannot even imagine to do. I would not, for instance, consider interactive 3D-visualization of a database. To give another example, if it is decided that we build a computer game in 3D, I have already reckoned what some of the problems are: for example, modellers and animators are needed rather than a HTML-coder. But why not architects? Why not model train builders? I got this inspiration while visiting the National Building Museum, where the local Washington DC Lego Train Club was showing off their extraordinary creations. I realized that when we are framing a design problem, we shouldn’t choose the material, medium or method first – this might limit our possible solutions.

When designers decide which method to use (i.e. how to approach the design work), they also perceive the design problem in a certain way. The method (whether Agile or User Centered Design or Activity Based Design) blinds the designer to some aspects and it highlights others. Methods are nevertheless necessary, but in order to get the whole picture I must recognize these blind spots from the outset, and perhaps explore the problem space using multiple methods. Otherwise the method is pressed upon the material and the problem,  and they get locked into being certain things.

when we are framing a design problem, we shouldn’t choose the material, medium or method first – this will limit our possible solutions

This has been a problem with the design of many applications and websites. The management decides on a method and it is imposed on the problem and onto the material. It doesn’t matter what the problem is, and it doesn’t matter what material the project is working with, they still use the same method ( probably one that is also trendy). However, it´s irrational to try to use the same development method in web store projects as in space shuttle projects. We should not think that there is one material and one method that works for all problem spaces, and this is especially important in the early ideation phase when we should be engaged in divergent thinking – how much divergence can their really be when the medium, material and method are already set?

Don’t get me wrong, I love the web. I hope to be working with the web in 10 years, in 20 years. But the web is just a canvas. Or perhaps a better metaphor is clay — raw material that we shape into experiences for people. But there are lots of materials — media — we can use to shape experiences. Saying user experience design is about digital media is rather like saying that sculpture is about the properties of clay. That’s not to say that an individual sculptor can’t dedicate themselves to really mastering clay. They can, and they do — just like many of you will always be really great at creating user experiences for the web.
~ Jesse James Garrett, IA Summit 2009 The Memphis Plenary

What is our clay? How do we explore and create great user experiences when we always go back to the same well, use the same materials and the same design method – how can we not always arrive at the same solutions?

When you have a hammer goes the old cliché. But what if you used clay? Legos?

In my view, a design method consists of a complex set of techniques tied together by a common, underlying philosophy. Every designer has a repertoire of methods and examples that make up his or her experience. The experience is of course tied to what sorts of projects the designer has been working on. The examples that a designer has seen influences how design problems are framed; they also embody the designer’s knowledge of the design materials. Someone who has worked only with web projects has a repertoire of examples from the web, but also has  knowledge in design materials like paper, Omnigraffle, XHTML, Photoshop, JavaScript, et cetera. As user experience professionals charged with designing information,  spaces,  and interactions between people, people and spaces;  people and objects;  and people and systems,  I think we need to step back and try other media, materials, and methods in the problem space exploration phase of our engagement. We shouldn’t limit ourselves to doing what we´ve always done, with the materials we´ve always used, according to a methodology handed down by management because it is the latest three-letter acronym.

“We shape our tools, and then our tools shape us.” McLuhan´s simple maxim is more relevant to today’s user experience designers than ever before. We need to explore new ways, new materials, and new methods if we really want to innovate and create fantastic, crafted, playful experiences that engage our audience.

Top image by batega

Lately I keep posting articles about augmented reality. And although I really like it, I have to admit that it still feels a bit quirky. Printing out a strange symbol and holding it in front of the camera is not the submersion people are looking for. Fortunately I came across a video that showed me that a true experience can be reached.

The video shows a photobook like any other. It doesn’t contain strange symbols with extreme contrast, but is realistic. When holding it in front of a camera magic seems to happen. I really like the simplicity of the graphics used. It’s not 3D, but kept simple… making it a nice experience. A great detail about this video is the camera. Notice how it is combined with a desk lamp, making it natural for people to aim at the part you are reading. This way you don’t have to hold up the book in front of the camera anymore. Really nie.

Hopefully this example will become a standard on which we can experiment further.

Handheld projectors are the next big thing in mobile devices. They make it possible to project a screen on almost any surfuce, anywhere. This technology opens up a whole new world of interactive possibilities. A group of researchers came up with the idea to explore multi user interaction for these handheld projectors.

At CES09 the handheld projectors were received with great enthusiasm. The researchers in the video are exploring the possibilities and complexities that arise when you combine multiple projections in a room and make them responsive to each other. Check it out:

The video shows a lot of explorations, not all of them interesting. But that’s how research goes. I personally think handheld projectors suck when you use them for highly interactive tasks, like swapping and sharing files. But they are interesting for exploring and playing around. The treasure hunt idea could be interesting though.

But just exploring technical possibilities isn’t interesting. It’s also very important to ask questions. Like: how do people feel and act when they suddenly can share their personal data in public? I’m really curious in what way people will use it. I don’t think they will beam their agenda and personal letters. In our minds these things are personal and for us. A mobile phone is small and can be ‘protected’ from the world, that makes it a great device. When people have a beamer we have to look at new possible uses. But these are just my thoughts. It would be great if the researchers would explore this. My hunch is that video projecting and new explorative games (the search light metaphor is great) such as the treasure hunt and augmented reality will make the projectors big.

Primal Source is an interactive performance/installation which has been created for an art festival in California. They used a large-scale outdoor waterscreen/mist projection system to project a stunning light-show onto the mist. The installation responds to sound emanating from the crowd, thereby letting the crowd create their own performace. That night 200.000 people brought it to life, by walking past the installation.

Different modes where displayed depending on how active the crowd was and some modes even created ‘creatures’ whose shape, color and movement all reacted to the crowd. Here is a video. You might want to turn down your volume a little bit since screaming is the first thing people tried to make it come to life.

Next to the fact that the installation is just so incredibly overwhelming and stunning to watch, it reminded me of something else which al together gave me an interesting perspective on the whole concept of interactive art/architecture.

Just recently I came across a man on television who was passionately explaining why the bow and arrow are such an important part of our history. He even claimed that without it, we would have never gotten to the moon. I can’t remember who he was since I came across it while flicking through the channels and I just watched it for a minute or two, but he made an interesting point. He saw the bow and arrow as the first time man exceeded/amplified their own physical abilities. Before that, spears and where used and we where limited to our own physical strength. Now if you look at it from that perspective, the bow and arrow are suddenly a much more interesting phenomenon (at least that’s what I found it to be).

What I see happening more and more nowadays within interactive art/architecture is that it amplifies things we haven’t been able to amplify before. A lot of them are starting to amplify the presence of people and sometimes they even react upon it. But what we’re also seeing now is that they’re trying to amplify emotions and expressions. To me, this is something very interesting. Especially because that’s an area which is still relatively undefined. What would happen if we would take this out of the museums and bring it into our daily lives? What will happen when we amplify our emotions and thereby making them visible to others? That’s a question which is occupying my head at the moment. I’m curious what you think about that.

Primal Source is an installation from Haque design + research.

By now we have seen a number of different ambient futures for the web here on Johnny, which almost all include some form of augmenting the physical world. Recently Pattie Maes of the MIT medialab presented Sixthsense, a wearable interface that augments the environment and uses gesture based input. A big difference with the things that we’ve seen here so far, is that this one actually works…

Sixthsense is the work of Pranav Mistry. I could give it try to explain to you how it actually works, but I’m sure Pattie Maes does a far better job than me so I’ll just leave you with the video.

Sixthsense is one of the first things (and one of the most impressive things as well) I’ve come across, which puts the data from the internet to good use in such an intuitive way. Off course the device itself, in its current form, is not something you’ll find on the shelfs any time soon. But it is one step closer to a future in which we are able to use any data, anywhere, in any way, and without having the feeling that we’re interacting with a computer.

I can’t wait till technology like this becomes available to a broader audience and gets accepted by society. When more people get access to a certain technology, the more interesting things tend to happen with it.

In the wake of the ever increasing popularity of gestural interfaces, Dan Saffer wrote his newest book: Designing Gestural Interfaces. In this publication by O’Reilly he tries to give us some valuable insights. But did he succeed?

Gestural interfaces are not new, but they are more popular than ever. Everywhere around us you see new ones popping up. And before everybody is coming up with their own interpretation and translation of what they should be and do, we must evaluate what has happened in the past and how we must deal with the future. Dan Saffer’s book is a great starting point for this exercise.

Let’s go through the book chapter by chapter.

The basics

The first chapter is an introduction to the subject. It gives a good and basic explanation. One interesting part of this chapter is ‘matching the gesture to the behavior’. This approach states that the best designs are those that ‘dissolve in behavior’. Here the product must become part of what the user is doing fluently. This is what good gestures should be, combining peoples natural behavior with a related action.

The human body

In the second chapter Saffer focuses on the human body, something a lot of designers often forget. The importance of understanding the mechanics of a human body is really fundamental in order to design good gestural interfaces. What is you body capable of doing? And what not? And we should not forget ergonomics. This is important even though there is no mouse. Saffer goes into this subject rather well, describing possible limitations and pointing us on some simple, but valuable facts (10% of adults are left-handed).

Patterns for touch, interactive & free form

After the first two chapters the book dives into possible patterns for gestural interfaces. It’s an important step Saffer tries to take here: trying to define an international set of patterns. Several companies are developing gestural interfaces and are creating their own patterns, independent of each other. Some, like Apple, are even trying to patent some patterns… so it’s really important to start describing a set that is recognized by users and freely usable by any company. It would be a great step if the patterns Saffer describes would be available online, open for discussion and growth.

Documenting gestural interfaces

Documenting static websites in the 90s was pretty straight forward. But with growing interactivity and gestural interfaces it is a challenge for designers to capture the way an environment or device interacts with a user. The fifth chapter of Designing Gestural Interfaces looks into different ways to document your project. The one I like the most is drawing storyboards, encapsulating not only the interface but also the context and how the user behaves in it. It’s a great way to test for yourself if the design works, but also a superb way of presenting the concept to clients.

Prototyping

When you’re done with the sketching you want to prototype an interface. In the sixth chapter of the book you get some low- and high-fidelity ways of prototyping. Amongst the low-fidelity is the creation of paper prototypes, which I really like. It’s an easy and really fast way of making a ‘working’ version of your product. You can give it to a colleague and let him play around with it. But even playing around with it yourself is really helpful, since you will ‘feel’ and see what it does. Of course paper prototyping is not always the best solution, since gestural interfaces can also be about bigger objects or even spaces. But these are also easily prototyped. Saffer gives some nice examples of these.

Communicating

For me chapter 7 “Communicating Interactive Gestures” was one of the more interesting ones. It learned me the true importance of communicating that there are interactive gestures or not. Imagine that there are some public spaces that have gestural interfaces, and some don’t… but it’s not communicated well. This will cause real stress, since people won’t know if, when, how and what they will trigger an event. It’s really important to be clear about this, stating when there is an interaction possible… what it triggers and how. But in order to achieve this we will have to find an international language, explaining this to us. It could be something as simple as an RSS icon, but it has to be something clear.

Future

In the last chapter Saffer tries to sketch his vision for the future. Unfortunately he plays on the safe side and only describes the main trends going on in the field. So when you follow the important sites in the IxD and UX field, you are covered and can skip this chapter.

Conclusion

Before I started reading this book I thought it would dive deep into the world of gestural interfaces. But I should have known better… Since this is the first serious attempt to capture the subject it is logical that it needs to introduce us to a lot of new things. And although only the first chapter is called ‘Introducing Interactive Gestures’ I have to say that the entire book is an introduction to the subject. This doesn’t mean it doesn’t give helpful information, but it never goes into the subject deep enough to totaly cover it. This makes Designing Gestural Interfaces a great book for people new to the subject, like students. Let’s take this book as the fundament and see some new ones building upon it, going more in-depth.

Book Details
Designing Gestural Interfaces
author: Dan Saffer
publisher: O’Reilly
details: 247 pages, paperback

For the last months we’ve seen dozens of augmented reality applications. The technology has become so low-level that more and more companies and individuals are playing around with it. This results in a lot of funny and (sometimes) interesting applications. But the low-levelness also caused a good side effect: virality.

First: We, designers, get to experience so many new technologies and possibilities that we are easily bored… This ‘curse of knowledge’ blinds us, making us not see how the audience will respond. For example: for us the iPhone is the most normal device in the world, multi touch is something we know and we are already looking forward to the next thing. But ‘the masses’ don’t experience it like this, for them it’s still a novelty… and most of them see it as unreachable for now. Augmented reality is quite similar.

Designers may be used to the technology behind augmented reality and understand how an icon on paper can be transformed into a 3D animation you can move yourself. But we must never forget how our target audience thinks. For ‘the masses’ augmented reality feels like magic. They don’t think in technologies, but just experience the result. And when they do this, they are overwhelmed by the coolness of augmented reality.

Check out these video from somebody experiencing augmented reality for the first time:

For me it’s really important to keep the above points in my mind. The ‘curse of knowledge’ is very strong and can make a designer fail totally. What triggered this thought was an article I read on theory.isthereason where the writer referred to the great number of home made videos on augmented reality. When looking at the different video’s you see how people perceive the result they get. They don’t know how it works, just that it is really cool. So cool, that loads of them decided to make a video of it and spread it on the internet. Freely promoting companies and products, like the GE campaign. In the case of the GE augmented reality stuff, it resulted in a viral campaign (which I am joining right now): check out the number of videos people uploaded. Amazing.

For me there are two lessons: let’s stay curious and not be blinded by the ‘curse of knowledge’. And the other one is more direct… augmented reality is a great starting point for a viral campaign, since it impresses people.

Yesterday I posted an article about personas and found out that it was the 99th post on Johnny Holland. So being a superb mathematician I concluded that the next article must be the 100th, which is a great opportunity to recap what we’ve been doing in the last months… and what we’d want to do in the future.Johnny started off in October 2008 with the goal to become “an open collective talking, sharing and finding answers about the interaction between people and products, systems or processes.” We wanted to create a place where we could be open to each other and actually ask question and be curious again. And since that moment a lot has happened.

Besides the hours I spend every night chatting, mailing, reading and writing for Johnny here are some facts: At the moment we have 28 registered Johnnies contributing to Johnny. We’ve grown +100 RSS feeds per month and right now +600 people are linked to it. On a daily base 200-400 people visit the website. Right now we’ve got 100 articles published, with an average of two responses per post. So we are growing steadily.

But facts aren’t everything… so I asked my fellow Johnnies what Johnny brought them until now. This is a great way to promote the coolness of becoming a Johnny, and I didn’t delete any negative responses (there weren’t any – w00t). A few responses:

As a blogger it’s also forced me to elaborate on and articulate my thoughts, rather than just sending around links. – Vicky Teinaki

For me, Johnny represents a conscious re-alignment of my thinking around UX and the disciplines that come together to deliver those experiences to people. [Johnny's discussions] are well and truly embedded in the language, theory and practice of design of UX. To be involved in that discussion is a nice change of pace for me. – Steve Baty

“Being a Johnny” doesn’t feel like anything in particular. But I think that might be b/c JH is not [yet] really a community. [Johnny] is probably one of my favorite outlets for “published” writing. I like how “free” it is compared to other sites. The approachability is key. – Dave Malouf

Goal: becoming a true collective

We’ve grown faster than I could have wished for, but we’re not there yet. One of the most important things we wanted to achieve was becoming a true collective. Two responses per post is nice, but not really a true collective. We want to search for the conversation between writer and reader, getting everybody involved. Writing stuff that makes you think and respond. Some articles where we found the discussion we want were Discovery vs Creation, Deconstructing Analysis Techniques and Why shouldn’t I kill personas (where the discussion was the food for the article). But we’re searching for any means to lower the bar and get you guys involved. So if you have any ideas or suggestions, let us know.

What does the future look like?

Besides becoming a true collective we’ve got a lot of dreams and goals for the future. And instead of brewing them in a black box we want to share them with you, so that you can get involved if you want. Here are a few brainfarts we came up with:

• Johnny TV: an online TV channel where we can share interviews and video columns
• Johnny event: a real event where you guys can listen to talks, but also get actively involved in discussions and creative sessions
• Johnny experiments: have an online place where all Johnnies can experiment with each other in an open way, sharing ideas and… experiments

Besides these three concrete ideas there were some other wishes. Some writers want to collaborate more in articles with others, making joint efforts. And one interesting and important direction is seeking the international character; publishing translations of non-English articles (Dutch, Swedish, Danish, Japanese, whatever). And last but not least: going to Interaction10 and meeting up with other Johnnies.

So stop lurking and begin discussing

Join Johnny (or something bad will happen to your Star Trek collection). This can be as an active writer, but just as well as an active responder. Give us your thoughts and opinion and feed your urge to learn. And for now, give us your input for the direction Johnny should take.

Over the last few years I’ve become more and more sceptical on the value of personas. I know they’ve always been a popular part in the user centred design methodology, which kind of means that they are holy. But I also believe that from the moment they were introduced they were also misused or based upon the wrong data. For me this was the moment to call in the help of a few experienced UX friends. Why should I still use personas?

For the discussion I e-mailed Steve Baty, Adrian Chan, Will Evans and Dennis Koks. This resulted in a four-day long mail discussion. Below is my translation and interpretation of that discussion, which proved to be of great value for me. Note: right before I finished this article there was an interesting discussion on the IxDA discussion list.

The way personas should be used

Before we dive into the discussion it’s important to look at personas from an encyclopedic perspective. What are they and what are they supposed to do? According to Alan Cooper, the grand creator himself, personas are and were originally intended as a heuristic. They are for client communication and orientation. As Adrian Chan states it “they encapsulate and personalize use cases.” To be able to do this you need to have quantitative and qualitative research data, which can be analyzed thoroughly and translated in worthy archetypes.

How are they often used?

Unfortunately the textbook description of personas is too often a myth. Everywhere I look they are NOT based upon thorough research and mainly a tool to please the client. Will Evans says “that a great many advocates of personas simply don’t get to do them – or do them right – because no money is allocated for design ethnography – and even if there is money dedicated to research, practitioners don’t know how to take that research and turn it into personas which are actionable in the actual design process.” I think his remark hits the spot. Maybe big companies such as Microsoft, Philips and Nike have huge budgets to do proper research, smaller companies and design companies often don’t have this luxurious position. But this doesn’t take away the need for proper data. We need to look for possible solutions here, which isn’t coming up with personas based upon assumptions and ‘experience’.

Adrian Chan, in my opinion a pioneer on the field of social interaction design, states “I’m all for a move among social interaction designers to replace cardboard user peronas and instead use psychologically-grounded personality types.” This could prove an interesting improvement. According to Adrian these personality types should not become archetypes, since these are usually not interaction oriented and not specific to social media. It’s an interesting thing to try and understand the personality types, expecially when they are based upon actual psychological research.

So at this point the scepsis was still in my head. Fortunately the discussion gave me three useful reasons why I should continue with personas. Let’s check them out:

1. Great for design decisions

One of the biggest advantages of having personas, based upon good research, is that you can use them for design decisions. As Will Evans puts it “Someone is going to make decisions – often a vocal stakeholder who thinks they know a lot more than they actually do – the VP of Marketing that wants to prioritize a certain feature or piece of functionality because “My daughter is our target audience – I know my daughter – so we should do X” and if a designer has no data, no research, and no personas derived from that – no matter how bad the idea – there is no way to argue against it. User research and resulting personas helps in making prioritization decisions about initiatives and features based on actual users as opposed to the whims of:
1. The designer with their own prejudices
2. The powerful stakeholder
3. The developer
4. Anyone who is not the intended audience”

2. Great as a debrief / briefing

When working for a client it’s really important to dive into their world and prove you really understand it. Personas can help you with this. They form a good summary of the target audience and are a great debrief for your customer. After presenting them I always start a discussion, trying to see how their world matches with the personas. Being truly interested in their audience, but also in how the client perceives it is worth so much. It brings you on the same level as the client, making them feel you really care about their message/product/service.

Another great thing, besides debriefing, is actual briefing. When fresh team members join the design team personas form a perfect guide into the project. They are a nice summary of the data, which gets the fresh people up to your knowledge level faster. This is the real goal of personas: being a means to translate and transfer research and knowledge to others. As Steve puts it “The persona provides us a way to transfer some of the value we derived from the research & analysis process to other members of the team. It isn’t a perfectly efficient transfer – some of the value is lost. That’s what we’re doing when we communicate *through* the artifact: transferring that knowledge/insight/learning out to people who couldn’t be involved in the process of producing it.”

Remember to approach personas from this perspective and it will help you make them more communicative. They should tell a story for themselves, dipping you into their mental and physical world.

The end result and the data is far LESS important than the immersion in the experience of the target audience – Will Evans

3. It’s about the process

I always learned to save the best for last. So this third reason was the one that made me a believer again. It was a quote from Steve Baty that made me see the light: “Personas are an outcome; not the process.” Everybody is staring at the outcome all the time, using the personas as if they are treasures… but that’s not it. It’s the process itself that’s priceless. Personas are only a tool that force us (designers) to do actual research. It’s a means to make us submerge in the world of our target audience and truly, deeply understand them. It helps Dennis Koks “to create a better understanding, and it helps [him] interpret the outcomes better during the rest of the design process.“ Steve Baty states that “only some of the value of personas is encapsulated in the end result; much of it comes from the exploration of the data itself.”

Personas are an outcome; not the process – Steve Baty

In the IxDA discussion Dave Malouf said that “if you are selling Personas then it seems that that is your first mistake. Sell research and don’t even tell people how you are going to model it. Maybe the research itself will tell you the appropriate way to model your analysis.”

So immersion is the key. In my mind it has even become one of the few user centred design approaches each designer must undergo. Will even wants to go so far “as a designer the whole goal is a violent sense of empathy for the user in their own context – the data in this case and the persona are completely useless if there is no bond with the end user. Design insights that were not directly experienced by the designer are worthless.”

“This raises an interesting question: do we need to carry out our own research? Is it enough to analyze the data? If we get a series of ‘design insights’, how bad is that?”  – Steve Baty

As a closure (in other words: I’m starting to repeat myself)

A thing about personas which, in my opinion, is still overrated… Is the value throughout the project. It is often believed that after personas are created they will live on and should come back in all discussions. Even when I’m typing this I want to believe it. I want to put their portraits on my desk and always use their names in discussions, but it almost never works out like this. For me they are important because they make me do research. When I create personas I have to force myself to trully understand the user and context. It makes me want to collect all the data and the translation into personas is more important than the end result. And after that the presentation of the personas to the client also helps to show and test if you really understand their customers. It’s a great starting point for trust with the client and forms a good discussion. After this I think they are less important for the people who actually created them, since they already submerged… they can use them as a reference, reminding themselves sometimes. But they are a great tool for new members of the design team.

I wish to thank Steve Baty, Adrian Chan, Will Evans and Dennis Koks for a great discussion. And Dave Malouf for stealing his quote. In my opinion this discussion is what Johnny is all about.

Images by moonsoleil & randy son of robert

Hylozoic Soil is an interactive architectural sculpture created by Philip Beesley, an architect based in Toronto. It can sense the presence of someone or something using proximity sensors and kinetic actuators, and responds with air movement. The result is an incredibly organic sculpture. So organic that the way people tend to interact with it, shows more resemblance with human-human interaction than human-computer interaction.

Here is a video of the sculpture (there is something wrong with the sound though):

Next to the fact that this sculpture is an incredibly impressive piece of work that interacts and moves in some of the most organic ways I’ve ever seen, it’s also interesting to look at how exactly people interact with it. People tend to move around it in a very careful manner and explore the sculpture using their hands and body movement without loosing their patience. It’s a form of human-computer interaction which we rarely see.

Today’s human-computer interaction is often subject to numerous expectations. If it doesn’t meet these expectations people tend to lose their patience and sometimes even get angry. I guess meeting these expectations is part of ‘user-centered design’. However, we have different expectations from human-human interaction than from human-computer interaction. What a project like Hylozoic Soil shows us is that the more organic human-computer interaction feels like, the more our expectations start resembling those from human-human interaction (in which we take the time to get to know each other, respect each other and often have more patience). A very interesting phenonomen.

More about the sculpture (including more video material) can be found here on Philip Beesley’s website.