The danger of doing the research wrong

No related posts.


Did it ever happen to you? Having a client that based his decisions upon the wrong research data, which made the project go in the other direction? I bet it happens to a lot of design teams. Today I came across a very funny video that illustrated this perfectly.

Doing the correct type of research and knowing how to do this is extremely important. In the example above you see how asking the wrong questions at the wrong time can be fatal. Focus groups are not the best method for testing if a product is good or not. This video reminded me once again that guiding a client in doing research is extremely important. Getting them to do the correct research and helping them how to do this will not only make their product better, it will also make your project better. But in order to get this done: be prepared.

I’ve had several meetings with clients that wanted to do research: the wrong research. They heard from an ‘experienced’ colleague what type would be best, how many people they needed, etc… And more than once they try to tell us that they can do the research themselves (because it saves money). Another thing we often hear is ‘We have a big network of customers which we get together in a focus group and ask if your concept is good.’ Of course this isn’t the way we want to go. But it’s not that bad. Actually it’s pretty great that they want to do research.

All we need to do is convince them what type of research should be done. And there starts the challenge of convincing a client who thinks he knows best. My 10 cent tip: refer to articles and books. I’ve had many discussions about the type of research and number of people you must use in it. But clients that have heard something from another colleague are difficult to convince. Of course I trust your talking skills, but having a book on the table or referring to some articles about research is the best thing you can do.

And here an extra video:

Top image by gerlos

Jeroen van Geel

Jeroen van Geel is founder/chief kahuna of Johnny Holland and the interaction director at Fabrique [brands, design & interaction], a Dutch multidisciplinary design agency. You can follow him on Twitter via @jeroenvangeel.

7 comments on this article

  1. Pingback: Interactions and Design » Blog Archive » Why Focus Groups Are Bad For Making Design Decisions

  2. Pingback: Twitted by jeroenvangeel

  3. Sometimes people rely on specific research approaches or a Focus Group because they are putting too much reliance on the one piece of research.

    They don’t have a rolling research plan.

    Suggest this is one reason we all get caught in this conversation.

    A rolling research plan says we are constantly listening to our customers and learning how to improve. It also suggests that are number of different methods are being used to look for insights.

    You may not be able to conduct the most optimal research but there is always an opportunity about different ways you can look forward (if people are listening of course :)

  4. Pingback: links for 2009-06-03 | burningCat

  5. Good article, you’re absolutely right!

    Most of the times those focus groups don’t work, because moderators ask the wrong questions in the wrong stage of the design process.

    Focus groups work only when participants are involved in an early stage. You can’t develop an product and arrange a focus group afterwards. This is done purely to justify what they did : “Our research shows that consumers like this idea”.

    I doesn’t work that way because participants are merely asked for their opinions about an idea. That isn’t doing good research.

    Instead developers and designers should involve participants before they have developed an idea for a product. That way you can ask the right questions and eventually come up with better product idea’s.

    (Better yet, let the participants a.k.a future users come up with the idea’s. They are the user-experts.)

  6. Marla Mitelman on

    This is great and an easy way to articulate how focus groups, can be dangerous if there’s no-one steering the discussion.

    I think it’s one thing to test that your solution works, it’s another to ask an open “what do you think” and expect to get back information that’s quantifiable.

    I’ll be sending this around to a few people … Thank you!

  7. Pingback: gerlos