The study,Isn’t It Cute: An Evolutionary Perspective of Baby-Schema Effects in Visual Product Designs‘, in the latest issue of the International Journal of Design looks at how consumers were affected by car fronts.

Non-manipulated (original) and manipulated (babyfaced) pictures of cars and faces.

They found that :

“consumers are sensitive to evolutionarily significant shapes in product designs. Based on our findings, designers can increase the affective value of products by creating cute designs which can benefit from the human predisposition to feel attracted by baby-schema cues (e.g., by emphasizing or exaggerating the features of visual key stimuli in product designs, such as very large headlights)”

This has been suggested before — for example, Scott McCloud illustrates beautifully how we see ourselves in the world — but it’s interesting to see how deeply ingrained our instincts are when it comes to looks. And of course, cute isn’t always appropriate. Luckily, another article in the same issue looks at more business-like products. As our own Jeroen van Geel tells us, it’s all about getting an appropriate product personality.

Still, if cute is on-brand for you, look at using baby like proportions on features. Is this why the internet is made of cats?

In-person user research used to be the only game in town, and as with most industry practices, its procedures were developed, refined, standardized, and then became entrenched in the corporate R&D product development cycle. Practically everything gets tested in a lab, hallway, or conference room nowadays: commercial web sites, professional and consumer software, even video games. But nowadays we’ve got remote usability testing.

This is a revised chapter from Rosenfeld Media's 'Remote Research'

Part of the appeal of formal lab research was that it provided a scientific-seeming basis for making decisions by using observational data, instead of someone’s error-prone gut instincts. Stakeholders appreciated the firm protocol and apparent reliability of properly managed lab research. But for all of us who have sat through formal studies with two-way mirrors, we know that there is a lot of bullshit that goes on—participants pretending to care, moderators pretending to understand, and stakeholders pretending to be open-minded. The appeal of what the kids call “guerilla testing”—informal testing, where you simply grab someone within shouting distance and ask them to use your interface—is clear. It’s easy, fast, and can produce great results. Lots of user research practitioners continue to use in-person methods because it’s what people have been doing for a long time.

There’s nothing specifically wrong with in-person research. But there is that whole Internet thing that’s been happening. It does have some unique properties we can take advantage of to do things that weren’t possible with old-school research. Like these things:

Insane Cost Savings
Usertesting.com is $39 per user. Compare that to flying to Chicago for three days to watch twelve people talk behind a two-way mirror, and that’s thousands of dollars in savings. Rolf Molich has been organizing the Comparative Usability Evaluation study (CUE) for eight years, where different usability methods and teams independently evaluate the same site. He knows something about comparing different research techniques, and makes the point that while there are advantages and disadvantages to a remote method like UserTesting.com, the “price/performance ratio was amazing” (that was before a price increase, but the cost is still quite low). Beyond travel expenses, other costs associated with in-person testing may be reduced or eliminated when you test remotely. Unless you’re doing guerrilla testing. With tools like Silverback, guerrilla in-person methods don’t have to cost much more than remote, but you are usually more limited by the audience. So in terms of cost comparison, let’s just say that remote testing will usually offer a big cost savings.

Time-aware research
Catching people in the middle of a task with a web or software intercept like ethnio (note: this is a product of ourselves) and calling them within a few seconds to share their screen and watch them use a tool remotely on their own timeline. It’s a degree of accuracy that never existed before. You could argue that it’s ethnographic in a way that is not possible with physical observation, but you could also spend your whole life arguing about that. Let’s not. Tools like Revelation and the Track Your Happiness project at Harvard use native timelines to gain insight. That’s a really big deal.

By now UX researchers are familiar with the importance of understanding the usage context of an interface—the physical environment where people are normally using an interface. Remote research opens the door to conducting research that also happens at the moment in people’s real lives when they’re performing a task of interest. This is possible because of live recruiting (the subject of Chapter 3 of the book), a method that allows you to instantly recruit people who are right in the middle of performing the task you’re interested in, using anything from the Web to text messages. Time-awareness in research makes all the difference in user motivation: it means that users are personally invested in what they’re doing because they’re doing it for their own reasons, not because you’re directing them to; they would have done it whether or not they were in your study.

Consider the difference between these two scenarios:

1. You’ve been recruited for some sort of computer study. The moderator shows you this online map Web app you’ve never heard of and asks you to use it to find some random place you’ve never heard of. This task is a little tricky, but since you’re sitting in this quiet lab and focusing—and you can’t collect your incentive check and leave until you finish—you figure it out eventually. Not so bad.
2. You’ve been planning a family vacation for months, but you’ve been busy at work so you procrastinated a bit on the planning, and now it’s the morning of the trip and you’re trying to quickly print out directions between finishing your packing and getting your kids packed. Your coworker told you about this MapTool Web site you’ve never used before, so you decide to give it a shot, and it’s not so bad—that is, until you get stuck because you can’t find the freaking button to print out the directions, and you’re supposed to leave in an hour, but you can’t until you print these damn directions, but your kids are jumping up and down on their suitcases and asking you where everything is. Why can’t they just make this stupid crap easy to use? Isn’t it obvious what’s wrong with it? Haven’t they ever seen a real person use it before?

Circumstances matter a lot in user research, and someone who’s using an interface in real life, for real purposes, is going to behave a lot differently—and give more accurate feedback—than someone who’s just being told to accomplish some little task to be able to collect an incentive check. Time-awareness is an important concept, so we’ll bring it up again throughout this book to demonstrate how the concept relates to different aspects of the remote research process (recruiting, moderating, and so on).

Circumstances matter a lot in user research, and someone who’s using an interface in real life, for real purposes, is going to behave a lot differently—and give more accurate feedback—than someone who’s just being told to accomplish some little task to be able to collect an incentive check.

Technological ecosystem
Some interfaces just don’t make any sense to test outside their intended usage environment. If you need the users to have their own photos and videos to use in a video editing tool, having them bring their laptop or media to a lab is an amazing hassle. Or, let’s say you’re testing a recipe Web site that guides users step-by-step through preparing a meal; it wouldn’t make much sense to take people out of their kitchen, where they’re unable to perform the task of interest. When this is the case, remote research is usually the most practical solution, unless the users also lack the necessary equipment. We also call this the participant’s “technological ecosystem” because it implies that their devices and computing environment have an impact on how they interact.

Democratization of user testing
That’s right, I said it. Democracy. As in, anyone in the world no matter how far removed from their potential audience can conduct user testing with less obstacles than before. After ten years of user research, 260 studies, and 3,000 participants at bolt | peters, we’ve noticed a trend lately that more people are doing their own research than ever before. And it’s great. There’s no reason to hire a specialist to observe real-world technology behavior. And that’s coming from a specialist.

Geographic Diversity
Even if you do have a lab, the users you want to talk to may not be able to get to it. This is actually the most common scenario: your interface, like most, is designed to be accessed and used all around the world, and you want to talk to users from around the world to get a range of perspectives. Will Chinese players like my video game? Is my online map widget intuitive even for users outside Silicon Valley? Big companies like Nokia and Microsoft are often able to conduct huge, ambitious research projects to address these questions, coordinating research projects in different labs around the world, flying researchers around in first class. If you don’t have the cash for an international longitudinal Gorillas-in-the-Mist project, then remote research is a no-brainer solution. If you can’t get to where your users are, test them remotely.

And Why Not?

Both in-person and remote UX research share the same broad purpose: to understand how people interact and behave with the interface you’ve made. There’s no need to set up a false opposition between the two approaches—one isn’t inherently better than the other. Despite the versatility of remote research, there are lots of reasons you might want to conduct an in-person study instead, most of which have to do with timing, security, equipment, or the type of interaction you want to have with participants.

Both in-person and remote UX research share the same broad purpose: to understand how people interact and behave with the interface you’ve made. There’s no need to set up a false opposition between the two approaches—one isn’t inherently better than the other.

Security
Security is often a concern for institutions like banks and hospitals, which deal in sensitive information, or companies concerned with guarding certain types of intellectual property. If you’re testing a top-secret prototype, you obviously don’t want to let people access something from their home computer, where it could be saved or screen-captured. On the other hand, you might also be doing a study on users who would be secretive about sharing what’s on their screen—government employees, doctors, or lab technicians, for instance. Either way, you’ll want to test users in a controlled lab environment to keep things confidential, especially if what you’re testing is so hush-hush that you’ve got to have your users sign a nondisclosure form.

Inability to use screen-sharing
You might also want to use a lab if your users are unable to share their screen over the Internet, for whatever reason. Some studies (of rural users, cybercafe patrons, etc.) may require you to talk to users who don’t have reliable high-speed Internet connections, who own computers too slow or unstable to use screen sharing services effectively, or who have operating systems incompatible with the screen sharing tools you’re using. These restrictions only apply to moderated studies, for which you need to see what’s on your users’ screens.

The need for special equipment
Depending on the interface you’re testing, you may require certain special software or physical equipment to run the study properly; this is most often the case with software that’s still under development. Getting users to install and configure tools to run elaborate software can be a pain (though that’s not unheard of), and requiring users to have certain equipment can make recruiting needlessly difficult.

The importance of seeing the user’s body
Some kinds of research will require you to study certain things about the user that are difficult to gather remotely. UX research has recently begun using eye-tracking studies, and for that kind of study, you’d need to bring the users to the eye-tracking device. Other studies might require you to attend to the participants’ physical movements, which may be difficult to capture with a stationary webcam.

Conclusion

You don’t necessarily have to choose between lab and remote methods. You can even conduct multiple studies on the same interface, using the findings from one study to add nuance to another. Probably excessive for the average study, but for really large-scale projects where you just want to gather every bit of information you can (a new version of a complex software program, an overhauled IA, etc.), being comprehensive can’t hurt.

You should have a good idea of whether or not remote research suits you. Give it a try—if it’s not your thing, you can always go back to lab testing. We won’t tell anyone.

In Johnny TV Features we’ll share with you interesting videos that we come across, enriched with our healthy opinion. This time we have ‘The Domestic Gubbins’, a video by Microsoft Research.

At first glance this video by Microsoft Research seems a bit whimsical or silly. But if we dig a bit deeper, we find there is a lot more to it, and there are some useful things that we can take away from it.

What is the video’s purpose?

The first thing to consider is what the video was for. What was its purpose? Once we understand this, we can go on to consider what the researchers got out of it and how this helped them in specific area that they were investigating.

So what was the video for? First, a bit of context. The Domestic Gubbins are part of a project called Objects Incognito (subtitle: Rethinking Machine Intelligence), which is “an ongoing enquiry into everyday ideas of intelligence”. We are promised a plethora of “intelligent” devices and technologies in the near future. But what is meant by intelligence, exactly, and what will it be like to live with these ubiquitous intelligent technologies? That is the question that this video attempts to answer.

Interviewing people

Why video, though? Jain and Taylor originally wanted to create the Gubbins as actual devices that they could give to people to live with and interact with. But this proved too challenging, so they decided instead to create this video, which shows how people might interact with the Gubbins. Then they showed it to people and interviewed them to find out what their thoughts and reactions were.

These interviews (excerpts of which can be seen here) provided the researchers with new insights and led them down new research paths.

How can we use this?

So how can we adapt and adopt this approach for use in our work? This research is very high-level and conceptual, whereas in our day-to-day work we usually deal with matters that are much more concrete. However, there are many situations where we would like to be able to put a product in users’ hands so that they can play with them (especially for products that are radically different from those currently in use or that address as-yet-unmet needs), but where we do not have anything close to a working prototype. In cases like this, we can use this approach to show how our new product might function and how people might use it, and then see how potential users react. We can then use the insights that we gain to guide the direction of our product.

What are the weaknesses of using an approach like this? The video itself has to strike the right balance between showing realistic scenarios of use and keeping things ambiguous enough to make viewers think and use their imaginations a bit. And when interviewing the viewers, a certain degree of finesse is needed to avoid drawing interviewees in a direction favored by the interviewer.

In conclusion, this is a very well-made video that we can borrow ideas from for certain situations (though definitely not all).

Johnny TV

This and many other UX videos are posted on Johnny TV. Should you come across a video that you think should be there, please contact us via mail or Tweet me @martinpolley.

All research methods have their pros and cons, the problem comes when you rely on just one method. I’m often disappointed when UX and IxD practitioners describe the research they do, and it’s obviously very one dimensional. They only do surveys, for example. Or they only do usability testing at the end of the project (it’s quite alarming but this practice does continue).

This problem isn’t restricted to UX and IxD of course, our marketing brethren might do likewise, referring only to Roy Morgan for insight, for example.

Each of these techniques can be incredibly useful for giving insight into a particular aspect of what you’re studying, but relying solely on one is a big mistake.

Introducing triangulation

This is where the concept of “triangulation” comes into its own. Also known as “mixed method” research, triangulation is the act of combining several research methods to study one thing. They overlap each other somewhat, being complimentary at times, contrary at others. This has the effect of balancing each method out and giving a richer and hopefully truer account.

[Triangulation is an] attempt to map out, or explain more fully, the richness and complexity of human behavior by studying it from more than one standpoint
- Cohen and Manion

The effect is rather like the use of telescope arrays, such as that shown in the photo above, which make use of many small telescopes spread over a large area to simulate the effect of one very large telescope. Not only does the array have the power of one very large telescope but it is more nimble, more practical and has the ability to cross-check itself. Because each individual telescope’s view overlaps that of its neighbours, the accuracy of each telescope can be validated to a certain extent by the others. This last property is a key benefit of triangulation in research, and one which we’re going to explore further.

Let’s start with a real life example of triangulation. Most of my own research projects of late, have included:

• Analytics
• Stakeholder interviews
• Interviews (face-to-face or phone as needs be)
• Cultural probe (aka diary study)
• Focus groups or workshops
• Secondary research (including an examination of market research data)
• Quantitative survey (to help validate findings with a much larger sample size)
• Usability testing (of existing product or early concepts)

Lots of different views, lots of data! Each method is used in a way which is appropriate for it, and when combined they allow a degree of cross checking. The above list would be applied to quite a large project; a small project might just include a competitor review, heuristic review and maybe a few interviews. Horses for courses, really.

Triangulation to minimize bias

Specifically, the problem with relying on just one method is to do with bias. There are several types of bias encountered in research, particularly the qualitative design research we use in the field of UX and IxD. And triangulation can help with most of them.

• Measurement bias – Measurement bias is caused by the way in which you collect data. Probably the most common form of this is the effect of the setting in which you conduct your research, for example, peer pressure on focus group participants. Triangulation allows you to combine individual and group research methods to help reduce this bias. Related to this is “response bias” in which participants tend to tell you what you want to hear. Again, a triangulated approach means you can combine self-reported and observational research methods to help balance out the problem.
• Sampling bias – Put simply, sampling bias is when you don’t cover all of the population you’re studying (omission bias) or you cover only some parts because it’s more convenient (inclusion bias). Some research methods make it easier to cover certain parts of the population, for example using phone interviews for interstate participants can be a good substitute for the face-to-face interviews you do with local participants. Similarly online surveys or cultural probes might make it easier for you to include geographically distant participants. Triangulation combines the different strengths of these methods to ensure you getting sufficient coverage.
• Procedural bias – Procedural bias occurs when participants are put under some kind of pressure to provide information. For example, doing “vox pop” style interrupt polls might catch the participants unaware and thus affect their answers. Similarly, an online exit survey might make the participant rush their answers to finish the survey quickly. Triangulation allows us to combine short engagements with longer engagements where participants have more time to give considered responses.

You can never rule bias or preconceptions but you should be cognizant of their presence and potential impact. In fact, failing to recognize bias is itself known as “design bias” (which also includes failing to disclose assumptions and possible bias when reporting your findings). Particularly with qualitative research, it is considered best practice to acknowledge bias and preconceptions. This is what Anthropologists and social scientists refer to as “reflexivity”.

In this regard, a bit of navel gazing is very important; self reflection and awareness of the limitations of your methods help you assess possible bias and take them into account when analysing data. Is my own sense of fashion affecting my perception of the fashionistas I am studying? Does my own view on smoking/drinking/gambling impact my research into marketing those products and services?

Tips for triangulation

Mix it up

Combine different techniques that balance each other out: quantitative vs qualitative, individual vs group, face-to-face vs remote, self-reported vs facilitated, short engagement vs long engagement etc. This is central to the idea of triangulation.

The right tool for the right job

It’s important to know what each is good for and ask the right questions in each. For example focus diary studies on what people do and think at a pertinent time, when you’re not around. Don’t ask them questions in the diary that you can ask directly (while still keeping the context). Likewise, don’t assess the usability of a website in a focus group.

Two heads are better than one

A kind of triangulation can also be achieved by having two (or more) people on the project. This helps immensely in terms of making observations, taking notes, analysis and “sensemaking”. Because unless the two researchers are very similar, they are likely to have quite different perspectives on what they are seeing and hearing, thus giving them different theoretical platforms from which to interpret and analyse. As well as simply allowing them to capture more data, the researchers balance each other out.

Layer upon layer

Yet another way of achieving a kind of triangulation is to conduct your research in successive layers of detail. Start off with a very broad piece of investigative research to identify top level issues and to provide better scope for the next layer. That next layer would be more detailed and focus on a smaller area than the first level. And so on and so forth.

Setup a feedback loop

Feedback findings into later methods to help validate or flesh out issues that have already popped up. Continuing the above example, during a focus group you might explore an issue that only one interviewee mentioned and see what the rest of the group thinks. Similarly, you can adapt your interview approach as you go, feeding back what you learn into the questions you ask. My favourite example of this would be using a quantitative method with a large sample size, such as a survey, to validate findings from earlier research using a relatively small sample size.

Be reflexive, grasshopper

Look for where preconceptions might exist, in yourself and your colleagues, and work out how you might be able to minimise their impact. As discussed above, using multiple researchers will make this easier, but if you’re working alone, keeping a journal or diary for yourself can work well in this regard. It forces you to examine what you’re doing and how you felt at the time—your emotional state on any given day actually makes a big difference to how you conduct research. And make sure you state a summary of this in your findings, along with any bias that you feel may have had an effect on the research.

Re-visit participants

Visiting the same people at multiple times throughout the research can give good results. For example, you might invite interviewees back for a focus group, allowing you to compare and contrast their views with other similar participants. This longer engagement with these individuals allows you to see how their goals, attitudes and behaviors change over time.

It’s often said that any research is better than no research, and largely this is true, but you need to take account of the fact that your methods may have limitations, such as bias. Triangulation is a very useful means of capturing more detail, but also of minimizing the effects of bias and ensuring a balanced research study, no matter how big or small that study may be.

Note: Patrick will be running a full day workshop on design research methods for UX practitioners at UX Australia 2009 – a 3-day user experience design conference, with inspiring and practical presentations , covering a range of topics about how to design great experiences for people. It will be held on 26-28 August 2009, in Canberra (Australia).

Top image by watchsmart

This time around we will look at videos that hardly involve any prototypes or scenarios. In these examples, designers are using video as a tool for research, inspiration and cultural probing. This article will probably be the thinnest in the series because examples like these are hard to find. But they needn’t be as these videos are technically less complex than all the rest we have seen.

Yellow Chair Stories

Yellow Chair Stories is a project by Anab Jain at the Royal College of Art. It is a “live service design intervention” conducted outside her apartment. She placed a yellow chair and a sign that read “My Wi-Fi network is open for neighbours and passersby. Free access from the yellow chair.” What resulted was an interesting and fun study on community, public spaces and network technology which became material for her to design future scenarios involving wi-fi.

Domestic Gubbins

“The Domestic Gubbins are a series of four fictional objects, designed as video probes, in order to enter in conversations with people around their everyday ideas of intelligence.” It is another project by Anab Jain with Alex Taylor at Microsoft Research, Cambridge. None of the objects from this project were functional, but the video prototypes were enough for their team to have meaningful conversations with people about such hypothetical products.

Microsoft Mojave Experiment

Although this video plays out like a viral marketing campaign, it’s interesting to see Microsoft combining video and user research to make a point about its products.

Snowbird Video Ethnography by Artefact

There is actually nothing exciting about this video, but I enjoyed it for what it is — a really nice example of video ethnography. Be sure to check out the Artefact blog where they have published the first of three articles about how to film customer insights.

Dispatch

I am currently working on a student project which is a service to help friends coordinate social activities with their mobiles phone. One way I have explored this area is by conducting small social experiments involving my own friends and their phones. Using video allowed me to document their experiences while gaining immediate feedback through interviews. Insights weren’t the only goal, I was also looking for stories, quotes or even footage that could be reused in fictional video scenarios when I present the final service at the end.

The ability to “play with the data” is a critical capability in analysis. We utilize this technique in many situations: searching for patterns or trends in our observations; or as another preparatory stage for further analysis. Sorting data in some way – alphabetic, chronological, complexity or numerical – is a form of manipulation.

(This article is the third part in the Deconstructing Analysis Techniques series.)

Manipulating data is that process of re-sorting, rearranging and otherwise moving your research data, without fundamentally changing it. This is used both as a preparatory technique – i.e. as a precursor to some other activity – or as a means of exploring the data as an analytic tool in its own right.

One of the key characteristics of a manipulation technique versus related techniques like transformation is that the underlying data remains unchanged. The main thing we’re doing is changing the relationship – logical or physical – that one piece of data has with another.

Reorganizing the data helps us to identify patterns that may otherwise not be apparent. In fact, it is almost certain that most patterns won’t be visible at first glance.

Let’s start by taking a more detailed look at some of the processes that contribute to the manipulation of data.

Re-sorting is literally a technique aimed at changing the order of the data. Re-sorting is most often carried out on numerical or quantitative data, but can just as easily be applied to text content. There are a few common types of sorting – numerical, alphabetical, chronological; as well as some that are much less common. For example, a list of responses to a survey question asking for a rating of a service might be sorted based on the severity and tone (positive or negative) of the review.

Sorting data helps to isolate significant individual values – the highest or lowest, most-frequent or least-frequent, first or last; and can also be a way of highlighting the shape of the data (more on this later).

Re-arranging is an activity that typically involves the physical or digital repositioning of a data element so that it sits in closer proximity to another. This might be to organize photographs into a narrative; or to juxtapose contrasting ideas for discussion.

Rearranging ideas through the manipulation of Post-its. Photo courtesy of Todd Warfel.

Much of the rearranging we do is exploratory, although at times it will be more directed. In these cases we might be trying to present a new configuration for our data – like rearranging furniture – to better support some activity.

Some of this manipulation will be more purposeful. We might be seeking to categorize a collection of photographs by grouping them into similar piles; or draw out common themes in user interviews. Recall, for example, in our article on Deconstruction we talked about breaking out key phrases or ideas into separate data points (on index cards, post-it notes etc).

What are we trying to achieve, though, with all this moving about?

Patterns

I’ve written previously on the important role pattens play in analysis; and the different types of patterns one might seek to find and identify in research data. The patterns we seek include:

• Trends: the gradual, general progression of data up or down;
• Repetitions: a series of values that repeat themselves;
• Cycles: a regularly recurring series of data;
• Feedback systems: a cycle that gets progressively bigger or smaller because of some influence;
• Clusters: a concentration of data or objects in one small area;
• Pathways: a sequential pattern of data;
• Gaps: an area devoid of observations;
• Exponential growth: rapidly increasing rate of growth;
• Diminishing returns: there is a decreasing rate of growth;
• Long tail: a pattern that rises steeply at the start, falls sharply, and then levels off over a large range of values.

Immersion

Many design researchers and design practitioners talk about the need to immerse themselves in the data before they can make any kind of sense of it. Manipulating the data is a way of gaining that immersion – that familiarity – through direct engagement.

Designers will undertake this process in a number of ways, depending on the format in which data has been stored. One of the most popular forms of manipulation is to write out key concepts, observations, and ideas onto Post-It notes and stick these to a wall.

The design team them actively moves the physical Post-It notes around, rearranging and grouping concepts and observations to help trigger creative ideas. This technique may be used in both the analysis and design processes to assist the design team, and there isn’t a write or wrong time at which it can be undertaken. This type of exploratory analysis can be powerful, and is a key tool in the card sorting analysis arsenal.

If running the card sort was the fun part, analysis is the painful part, at least until you get going. Exploratory analysis is like playing in the data – looking for connections that make you think “hey, that’s interesting”, or that show patterns of behaviour. – Donna Spencer, Card Sorting

Donna’s quote highlights two important characteristics of this analysis technique: firstly, that it can help uncover and highlight key insights in the design research data; and secondly, that sometimes starting is the hardest part.

Where Do I Begin?

Design research – any research activity, really – can result in a body of data that simply feels overwhelming. Thousands of sticky notes containing observations or notes, covering the walls of a ‘war room’. Perhaps it’s thousands of survey responses, or dozens of interview transcripts. It may be hundreds of photographs taken of users in context; or hours of video of a user testing study.

Sometimes this richness of available data works against us, making it difficult to understand where we should begin. Like it’s counter-part in analysis – Deconstruction – the techniques of Manipulation are easy to undertake, and require little or no preparation.

Perhaps more importantly, Manipulation encourages exploration. It works well as an unstructured activity and therefore works well as an entry point into those vast collections of messy data points we’re so often faced with early in the analysis. If you’re not sure where to begin, begin with manipulation – the more tangible and tactile the better.

Uses of Manipulation

Despite the simplicity of manipulation as a technique, it delivers the heart of some very powerful analytic methods. For example, affinity diagramming is requires little more than manipulation (and perhaps deconstruction as a preparatory technique) to produce some real insights.

In many respects, the method of creating a mental model introduced by Indi Young in her book of the same name is another example of manipulating data with intent. Throughout the method data is manipulated – usually physically – through the use of sticky notes or index cards. Ideas are grouped and compared, collated or isolated, by physically repositioning and rearranging the physical object.

Manipulation can also be used to answer specific research questions. We can sort our data chronologically to find the first occurrence of an event. We can sort the data numerically to identify the highest or lowest values, or to identify the median figure (the middle observation) in a series of observations.

Perhaps we’ve already gone through an exercise of aggregating data points and tallying up the number of occurrences of each. We can now manipulate the data and sort in either ascending or descending order to identify the most common or least common responses. This combination of techniques – aggregation and manipulation – provides for an unsophisticated, but still useful ‘method’.

And that, of course, is one of the key things about each of the analysis techniques discussed in this series: whilst each is useful on its own, their real power comes from the ways in which they are combined to form the sophisticated and rich methods we tend to encounter in books.

Challenges

One of the greatest challenges we face when we start to play with our research data is a tendency to settle on the first arrangement; the first patterns; the first grouping. We begin with such a chaotic mess, that first glimpse of something that presents us with a clear view – some sense of real meaning – can be quite powerful. We resist the step of re-shuffling and messing it up again, and may therefore miss the opportunity to see a second, third or fourth pattern.

Another major challenge – which we’ve mentioned above – is that the volume of data can be quite daunting. As much as it is a good step to just get started, some sense of how data elements can be grouped or arranged before you begin is important; but it shouldn’t be an obstacle.

And, of course, we must be in a position to easily manipulate the data we’ve collected. This means the format and medium in which our data is recorded is critical. Storing data digitally is not necessarily advantageous or preferable: many designers will attest to the positive effects that can come from physically interacting with the data.

Summary

Manipulation can therefore be seen as one of many low level analysis techniques with which we work every day. We’ve all encountered it in one form or another, and probably spent little time considering it. And yet it is one of the major workhorses of any analysis effort, and one which we should understand.

When looking at a product have you ever heard yourself saying “why would anyone buy this?” or “why would people use this?” – I have. Unfortunately, there have been many times when I look at products and experience the “I don’t get it moment”. I mean, I understand the functions being demonstrated but I don’t understand what problem the product team is trying to solve. So we put our doubts aside and start working on the product anyway as you know at some point you can help simplify, redesign and make usability improvements. But something still niggles at you. What is missing?

Understanding value

The problem is we don’t spend enough time up front on projects discussing, assessing, defining and refining the value of what we make. We jump too quickly into design and build before applying rigor to what we make. Its easy to get lost in the product detail: a screen, code and forget what the product’s value is and who you are building it for. Everything we do should be to help move the product a little closer to success. Every question we ask, every piece of research we do, every design or sketch we make, every product walk through we have with stakeholders, should all help iterate towards understanding the product value – the copy, a widget, a function, a screen, the product framework, the product, the product line and where that product line lives in and around other products in the company and the marketplace should say something about its value.

we don’t spend enough time up front on projects discussing, assessing, defining and refining the value of what we make.

Stop and assess

We must dedicate more time up front, at the start of any project or before we jump into developing a new feature, feature set or redesign effort to better assess the value of stuff we make. What makes people want something in the first place, use it, continue to use it, buy more of the same, treasure it and keep it? The following list is by no means exhaustive, rather it attempts to get to the heart of why something is valuable:

We should…

• do our homework and investigate how product decisions are made;
• show how we can help the team make better product decisions;
• provide research methods to test assumptions along the way;
• ask: what does this product do? (what makes this product tick?);
• ask: what do you love about the product? (why would you buy it?);
• ask: what does the product team love about the product? (are they passionate about what they are working on?);
• ask: what does sales love about the product? (are we helping them sell more?);
• ask: could you sell the product? (if you were tasked to sell the product could you? Would you want to face customers with the product you have today?);
• ask: what features would you sell? (any stand out features? any useless features? any features you would lead with when selling?);
• ask: what are customers saying about the product? (does it really help them?);
• ask: at what point would you want to throw the product away? (at what point does the product lose its value?);
• ask: at what point would you want to upgrade? (what would you base your decision on?);
• ask: how do you want the product to shine in the market place? (what would make it stand out?).

By asking questions about the product and its value you are by doing this in fact demonstrating value. Your role is to test assumptions and ensure that you provide clear value for users and determine what deeper research is needed. That is the sweet spot – providing ways for us to manage, facilitate, guide and educate product teams to take the necessary time up front and at every stage to deliver value as we drive towards product success. We don’t do this enough and the product team often does not have a shared set of design principles, philosophies or design tenets to hold onto as the product develops.

Dont Ask Permission

Ask yourself, is there general agreement on the team about the product’s value? Is this ever defined? Don’t ask for permission. You are all in the right position now to question the value of what you work on and to help improve stuff. Questioning, improving upon and nailing down the value of something helps set our strategy in the right direction, helps us focus on building the right stuff and helps avoid storms ahead. So get clarity around:

1. Understanding the value of what we do
2. Understanding the value of the stuff we work on
3. Showing the value of what we do to others
4. Appreciating the value of the people who end up using the products we make
5. Appreciating and leveraging on the value of the product team skills available to help make better stuff
6. Assessing and finding value up front before we start making stuff

So what stuff do you value in your life and why?

I look forward to an ongoing discussion with you and to learn from your successes and failures.

Over the last years I have noticed that many books and articles talk about the usefulness (or not) of personas, delving a little into the actual production and design of the persona as well as defending it’s usage. Very few explicitly define some of the activities that occur within the design research phase. It was Jared Spool that mentioned the real value of personas being the actual process of engaging with users and developing empathy towards their circumstances and experience interacting with a product.1 The following article grew out of a conversation with Nathan Curtis of Eight Shapes (author of “Modular Web Design“) when I offered to contribute what I called a “Mood Map” to the Unify Documentation System. Let’s start.

Personas are to Persona Descriptions as Vacations are to Souvenir Picture Albums.

While people who didn’t go on the vacation can look through the album and think, “Boy, that must’ve been fun,” they’ll never get the full experience of what the actual vacation experience was. The album is just a remnant.
JM Spool, Personas are NOT a Document

The purpose of a Mood Map is to document and map the emotional states of a user [over time] so that it can guide the creation and communication of personas to stakeholders whilst also informing the design process itself. I’m not one for UX deliverables for their own sake, but this is one that carries a lot of weight with clients and also goes a ways towards offering ‘traceability’ for your personas.

This article will begin with a brief overview of design research, an overview of Mood Maps, when to use them, as well as when not. I will not address interpretive, phenomenological, or constructivist paradigms and how those may shape our views on design research or the particular tactics used to uncover user emotive states.

Design Ethnography

Design Ethnography is usually conducted to gain a deep understanding of the client’s target market in order to apply a customer-centered approach to the strategic development of the client’s brand in the context of a complex dynamic ecosystem that borders on chaos. In addition, ethnographic research seeks to reveal insights into how the target market shares information about their problem space and potential solutions with their immediate social cohort.

Design ethnography takes the position than human behavior and the ways in which people construct and make meaning of their worlds and their lives are highly variable, locally specific as well as intersubjectively reflexive. One primary difference between ethnography and other methods of user research is that ethnography assumes that we must first discover what people actually do, the reasons they give for doing it, and just as importantly, how they feel while doing it, before we can assign to their actions and behaviors interpretations drawn from our own experiences.

Hassenzahl and Tractinsky, in User Experience – a Research Agenda state that “It has become obvious that the design for user experience needs to aim to satisfy human needs beyond the merely instrumental, and to focus on how to create positive experiences rather than just prevent usability problems.”2 In other words, the aim of experience design is not only to serve our practical needs and to help us reach practical goals, but also to give meaning and to contribute to the quality of our life.3

Besides taking into account the human needs, we must consider the affective and emotional aspects of the interaction, and the full nature of experience must be understood to capture the essence of user experience before we can undertake the task of designing a better, more emotionally positive experience.4

Findings from a design ethnography project will influence both near-term problem setting and experience design activities, as well as longer-term dynamic mediated social-systems development. During such study I seek to uncover pertinent insights about the target market’s experience enframing their goals, objectives, and perspectives as it directly relates to the client’s brand; and the role that these activities play with regards to interactions with their environment including context, family, friends, group, community and society.

Design Research & Mood Maps

By Design Research, I specifically mean in-situ interviews and observation sessions which are conducted to probe deeply into the lives, habits, and emotions of target consumers as it relates to a specific product or service. A cross-section of participants of a robust enough sample size must take part in the various activities to gain deeper understanding and to move beyond ‘design-by-anecdote;’ to elicit key joy and pain points that occur whilst these activities take place in context experiencing the brand in solving real life problems.

While there are a number of tactical activities a design researcher can engage in including interviews, journals, usability testing, focus groups, and task analysis (‘Doc’ Baty’s article in UX Matters is excellent: “User Research for Personas and Other Audience Models“) – one that is particularly good at gaining insight into the emotive aspects of a user’s experience is the Mood Map. It is important to remember that Mood Maps are an intermediate deliverable meant to provide meaningful insight for the creation of personas, not a final artifact. You may also choose to never show these to key stakeholders, but only include them in the appendix of a findings document after the research phase is done.

Another important point is that Mood Maps are best used for larger, more complicated user engagements or scenarios, not small directed tasks – logging into an application would not be an appropriate use of Mood Maps.

Phases and Emotions

The diagram above describes the emotional ups and downs identified by study participants as part of the design exercise conducted during in-home visits with participants. Note that the location of the study is less relevant than the importance of observing the participants in the most likely context in which they will engage in their experience with the brand’s product or service. During the exercise, participants are asked to name each of the phases they went through from framing their problem through exploration and finally (hopefully) problem solving, and to then assign a corresponding emotion to each phase.

The diagram represents an average of participant responses. The exercise tends to uncover some important variations based on a number of factors, including each participant’s individual personality, profile, as well as emotional relationship with the brand – or a competitor’s. These variations are described in the “participants’ emotions” section for each phase which the researcher is encouraged to heavily document, photograph, and take notes.

Cycle of Exploration

Exploration is not a linear state, but rather a cycle of activities such as “imagine,” “research,” or “try-on,” each with a particular cognitive posture (I encourage you to identify more, for instance “ask,” “validation seeking,” as potential social postures a user could engage in).

It is important to reflect upon each of the phases of the user engagement and attempt to identify the dominant activity. During a study of this type people instinctively begin to combat the uncertainty of indecision by considering the circumstances of their goal and limiting their options based on various contextual constraints – the term ‘satisficing‘ is used to describe this. If it is possible to have the participants verbalize their thought process, it will aid in providing you with a richer understanding of their emotional reaction to a particular phase. These verbalizations should be captured and presented with Mood Maps made for each participant, some of which may end up in the personas as guiding insights for design consideration.

In presenting the findings, it is important to tell a complete narrative based on an aggregation of the findings before delving into particular anecdotes about any specific participant. An aggregate view uncovers both the joyous as well as the frustrating aspects of the interactions, which may highlight unknown, or at the very least, un-discovered, weaknesses in the user experience which can be marked for further exploration.

Addendum

By way of my friend ‘Doc’ Baty, I stumbled upon a blog post by Bruce Timkin which shows another way to visualize the aggregated Mood Maps: an Experience Wheel, like the one he found at Lego. Although it is unclear what research, activities, or methods are used to arrive at the Experience Wheel it’s still an interesting way to visualize the total user experience in phases.

Resources

1. Chapman, J. Emotionally Durable Design: Objects, Experiences and Empathy. Earthscan Ltd, UK, 2005.

2. Hassenzahl, M. and Tractinsky, N.. User Experience – a Research Agenda”. Behaviour and Information Technology 25, 2, 91-97, (2006).

3. Hassenzahl, M. and Roto, V. Being and doing: A perspective on User Experience and its measurement. Interfaces, 72, 10-12, (2007).

4. Desmet, P.M.A. Designing Emotions (PhD dissertation) Delft: Delft University of Technology, 2002.

• Observing the User Experience: A Practitioner’s Guide to User Research
  By Mike Kuniavsky
• Design Research: Methods and Perspectives
  by Brenda Laurel (Editor), Peter Lunenfeld (Preface)
• Designing and Conducting Ethnographic Research (Ethnographer’s Toolkit , Vol 1)
  by Margaret Diane LeCompte
• LEGO’s Building Block For Good Experiences
  post by Bruce Temkin

Being open-minded when conducting user research is almost the definition of the activity. In reality many people find it difficult to fully conceal their hypotheses, and the way they both ask questions and listen to answers ultimately limits what they’re able to learn from the research. Recently we interviewed a young man who had moved back to the US from working overseas and was now back in school. My colleague asked a question that was based on the framework of “old Alex” and “new Alex.” But of course Alex (not his real name) hadn’t told us that he viewed his transition in those terms. And Alex didn’t dissuade us from that framework until I explicitly asked him if he thought of himself as having a new and old version. A lack of open-mindedness can also impact how research data is analyzed.

In the design process, there’s a tremendous need for open-mindedness when considering solutions.

For example, a few years ago we worked with a technology company that had done their own ethnographic research into how enterprise customers were purchasing technology. They organized their results according to their pre-existing purchase decision model, rather than allowing the findings themselves to shape the model. In both cases, the learning is skewed by these presumptions, and the ability to act in a new way is limited by them.

In the design process, there’s a tremendous need for open-mindedness when considering solutions. Although design projects invariably have constraints, we’ve found real power in suspending those constraints when ideating. Even if a team is developing a new website, rapidly brainstorming concepts that impact hardware, branding, marketing, and beyond is not a waste of time. If those concepts are outside the brief, they may better illustrate the qualities of an excellent solution that can then be brought back into the (say) website. And there’s always the possibility that a great idea will expand the brief. It’s always exciting to encourage a team to brainstorm beyond their nominal solution area and see an organization rally to support them: we’ve helped hardware teams identify software opportunities, and Internet services teams identify retail opportunities. And there’s typically someone in the meeting who’s empowered to run with these “extra” opportunities and see if they can’t be developed further.

Living With An Open Mind

Chatting over dinner with some colleagues recently, at one point I bought up The Hipster PDA. One of the younger people at the table hadn’t heard about this, and once we explained the concept (essentially a binder clip and a stack of Post-It notes) she shook her head in gentle disgust and said “People have too much time on their hands.” I felt stung, stymied, and frustrated. Reflecting later, my frustration was not in the dismissal of the idea (because there’s a lot of crappy ideas out there) but in the dismissal of the people who had come up with the idea; essentially, throwing out the baby with the bathwater.

I’m sure that we can each recall ideas (and the people that created them) that we’ve dismissed out of hand. Malcolm Gladwell’s Blink has led to the inappropriate fetishization of rapid assessments (take to heart Peter Merholz’s summary of the book: “Snap judgments are valuable. Except when they are not.”). In “Living In the Overlap” (interactions, September/October 2008) I argued for more a considered approach to how we dismiss what doesn’t appeal: “Here’s a bunch of stuff I haven’t tried: Project Runway, High School Musical, American Pie movies, robot wars, molecular gastronomy, Halo 3, Dancing With the Stars, Frisky Dingo, sudoku, biopics, House, Desperate Housewives, Portishead, Fifty Cent, Dane Cook, The Da Vinci Code, The Life of Pi, Marley & Me, The Lovely Bones, Augusten Burroughs, and Mitch Albom. I’m mildly curious in some; intensely disinterested in others. A lot of it might make a “sophisticated” individual uncomfortable. But my profession is identifying and establishing the connections between people, culture, brands, stories and products, and that means it’s absolutely crucial that I know of, and a little about, all sorts of stuff that I may personally regard as crap.”

Malcolm Gladwell’s Blink has led to the inappropriate fetishization of rapid assessments

Of course, open-mindedness and curiosity are essential for innovators: designers, researchers, people-who-make-stuff-for-others. But an open-minded discussion about our own open-mindedness may be hampered by two factors. First, we tend to frame open-mindedness and curiosity as character traits, something that’s baked into who we are and won’t significantly change (at least once we hit our mid-twenties and are less prone to self-redefinition). Second, who would acknowledge to themselves, or others, that they aren’t open-minded? No one thinks they are closed-minded, just like no one thinks they are racist. We place a cultural premium on our personal qualities and that can blind us from taking too close of a look at ourselves. While I’ll try not to get too therapy-y here, one way we can improve is to learn to hear those moments in ourselves when we aren’t open-minded.

Exercises to Open the Mind

Let’s instead consider open-mindedness and curiosity as skills that can be developed. One way to increase our openness to new ideas is through improv training, which emphasizes accepting offers. Many people are now familiar with improv’s “Yes, and…” philosophy. It’s become an icon for successful brainstorming, where the reaction to a new idea (an offer) is always yes (an acceptance). In improv training, there are a range drills to build a natural facility for acceptance. In one example, actors line up beside the stage. The first actor takes the stage, and the next actor enters the stage, states who they are, and pretends to give an object to the first actor. The first actor accepts this gift, and uses it to explain their exit. The next actor comes onstage and the drill proceeds. The drill might look something like this:

Actor 2 enters
Actor 2: Hi, I’m a baker. Here’s a loaf of bread.
Actor 1: Thanks, I’m going to go put this in the cupboard.
Actor 1 exits
Actor 3 enters
Actor 3: Hi, I’m a spaceman and I’ve brought you this moon rock.
Actor 2: Thanks, I’ll take this over to the Smithsonian.
Actor 2 exits
repeat

It’s not funny, but the goal is not entertainment; it’s to reinforce a crucial principal. The training builds from there, with more complex games that develop a broader set of skills, but the fundamental driver is being comfortable accepting any offer.

Getting Out of the Comfort Zone

A corollary to open-mindedness is our comfort zones. It’s important to learn to feel the edges of our own comfort zones. Only then are we empowered to make explicit choices about whether or not to venture outside that zone. In Amsterdam recently I walked into a cheese shop, stocked with an array of huge wheels of cheese, and I became overwhelmed, and felt unsure how to proceed. I just wanted to sample Dutch cheese, not do my weekly cheese shopping (as I imagined was the typical scenario). Suddenly this was too much to navigate, and as an introvert, I wasn’t up for asking for help if I didn’t even know what to ask, so I walked out. As I left, I realized that I was lletting the opportunity pass by and I returned to buy some cheese [You'll either recognize exactly the small intimidation I'm describing, or else think I'm crazy, in which case I suggest you read about introverted travel here.]

I asked for a recommendation. The cheese-man asked me one or two questions, then handed me a piece of Old cheese. The cheese, of course, was delicious, and so was the empowerment of overcoming discomfort. The awkwardness I felt was about the complexity of trying to perform a familiar task in an unfamiliar context, and with the myriad options that introduces. Once I realized that and chose to ask for help, I was able to stretch my comfort zone just that much more. And a few days later in Leuven, Belgium, I reused this new script and once again bought cheese (a Bruggian version of Gouda).

A few months earlier, however, I gazed at the edge of my comfort zone and decided not to cross: walking through Santa Monica we came upon the Independent Spirit Awards ceremony. Crowds of people were gathered, waiting for a glimpse of the stars. We found the serious autograph hounds who were there with portable plastic bins stuffed with headshots for signing (and reselling on eBay). It was a definite subculture: people filled each other in about the unspoken rules: what happens when a celeb approaches, when to use your Sharpie, how to hand it to them, and so on. I was fascinated but my obvious outsider/passerby status felt like a barrier. And then I saw a woman covered in tattoos, where each tattoo was a signature. I realized her particular shtick was to get autographs and then go directly to the tattoo parlor to have that autograph made permanent (the ultimate version of “I’ll never wash this hand again!”). I watched her and the group for a while, and thought about whether or not I would ask her for a picture. There was something slightly wild about her and I couldn’t figure out how to make the request. Sure, in the cold light of these pixels, it’s easy to think “What’s the worst that could happen?” but in the moment itself we may deal with it less rationally. I was actively taking pictures during my trip and I really wanted a picture of this woman but I was never able to do it. As with the cheese, I did step outside the experience for a moment, look at where I wanted to go, and decide whether I was able to cross that gulf. Here, I couldn’t.

While I’m disappointed with the result, I have to acknowledge my own human limitations, and point out that both were deliberate choices about how to deal with the edges of my comfort zone. Looking at both situations, I can see how the presence of a familiar “script” where each “actor” knew their role (i.e., man enters a cheese shop) made all the difference to me.

Being open-minded and curious is a journey, not a destination. There are always new opportunities to grow and develop our abilities in the course of everyday life and our work. And on that journey is a bounty of new adventures, new stories, and new ideas. I believe that developing skills in building and managing our own open-mindedness and curiosity in everyday life is the first place to start in our development.

Top image: nogoodreason

Did it ever happen to you? Having a client that based his decisions upon the wrong research data, which made the project go in the other direction? I bet it happens to a lot of design teams. Today I came across a very funny video that illustrated this perfectly.

Doing the correct type of research and knowing how to do this is extremely important. In the example above you see how asking the wrong questions at the wrong time can be fatal. Focus groups are not the best method for testing if a product is good or not. This video reminded me once again that guiding a client in doing research is extremely important. Getting them to do the correct research and helping them how to do this will not only make their product better, it will also make your project better. But in order to get this done: be prepared.

I’ve had several meetings with clients that wanted to do research: the wrong research. They heard from an ‘experienced’ colleague what type would be best, how many people they needed, etc… And more than once they try to tell us that they can do the research themselves (because it saves money). Another thing we often hear is ‘We have a big network of customers which we get together in a focus group and ask if your concept is good.’ Of course this isn’t the way we want to go. But it’s not that bad. Actually it’s pretty great that they want to do research.

All we need to do is convince them what type of research should be done. And there starts the challenge of convincing a client who thinks he knows best. My 10 cent tip: refer to articles and books. I’ve had many discussions about the type of research and number of people you must use in it. But clients that have heard something from another colleague are difficult to convince. Of course I trust your talking skills, but having a book on the table or referring to some articles about research is the best thing you can do.

And here an extra video:

Top image by gerlos