My Recommendation: Stop Making Design Recommendations

Related posts:


It’s easy to believe them when clients ask us, designers, to make recommendations. We want to believe they love us for our wisdom, knowledge, and experience. They want our advice. And we love giving them advice. It makes us feel smart—like they finally “get” what we’re about. They want to do the right thing and we know how to help them. So, why is it bad to make design recommendations? They want it. We want it. Why shouldn’t we make the recommendations they’re asking us to give?

Simple: The recommendations don’t work. We end up looking bad. Clients lose faith in our skills. And the design doesn’t get better. Interestingly, in our research, the best teams don’t use recommendations. Instead they use an experimentation approach.

Patient, flexing his arm: “Doctor, doctor! It hurts when I do this.”
Doctor, checking the patient: “Hmmm. Well, I recommend you stop doing that.”

The Easy Out

Making recommendations is an easy out. You say, “Do this. Change that.” then wipe your hands clean of it. If they don’t do it, they’re obviously idiots. If they do, you’re brilliant. The best case scenario is they follow your great recommendation and it improves the design. But it turns out, that only one out of four possible outcomes.

They follow your recommendation and the design improves They don’t follow your recommendation and the design improves
They follow your recommendation and the design doesn’t improve (or it degrades) They don’t follow your recommendation and the design doesn’t improve (or it degrades)

What happens if they follow your recommendations and it doesn’t improve the design? What happens when they choose to not follow your recommendations and the design improves anyways? In either case, your future attempts to work with them becomes more difficult.

Changes cost resources. If the design doesn’t improve, then the organization has spent energy, money, and time on something that didn’t pay off. Are we considering that when we put the recommendations on the table?

Playing “Bet Your Salary”

UX Researcher Extraordinaire, Meghan Ede, has a rule of thumb she applies to her research team’s recommendations. The team members can only submit a recommendation if they’d be ready to put a full year’s salary down as a guarantee that the design will show improvement.

Would you be willing to do what Meghan does with your next set of recommendations? Go ahead: take out your checkbook. Write out a check for your take-home salary, after taxes. Pass it in with your recommendations, while telling them that, if the design doesn’t improve, they can cash the check. How confident are you feeling about those recommendations?

The Experimentation Approach

What our preliminary research has found is a typical recommendation looks something like this: “Users had trouble seeing the field labels. I recommend you put the label on the top of each field, instead of on the left.”

However, some teams are using a different approach: “We’re seeing that our users have trouble with the field labels. We’d like to try an experiment and see if moving the labels to the top of each field makes an improvement.”

It’s a subtle difference. And it was the approach we saw most in use amongst those UX professionals who had a solid track record of consistently improving their designs. These professionals told us they refuse to make recommendations, but love to experiment.

Discussing the Meaning of the Observations

I found the process from these high-performance teams quite interesting. It starts with a team discussion of the underlying observations and what it means. The team explores all the different interpretations. “Is it possible the users didn’t see the labels because they are too far away? If the font hard to read? Are the users not recognizing the terms? Were we measuring the wrong tasks?”

Then the team guides the conversation to other research that may fill in any holes, group discussion of alternatives, and measures to signal when the users’ behaviors change in the right direction. Often, this is followed by further research, then more discussion.

This process is very different from the recommendation approach, where the local UX expert makes a pass at the design and puts together a list of things that need changing. Instead of putting the onus on someone to come up with winning solutions, the entire team pushes the design into improvement, one experiment at a time. Some changes will work as intended, others won’t, but with each change the team learns something.

The result is that the entire team becomes better informed about the design they are building. No one person carries the burden of improving the design. Nobody has to be in the position of being all-knowing, always right. Changes are not seen as final, but as an ongoing process of improvement.

A Change in Mindset

Making the move away from recommendations is very hard. As I said, making recommendations is the easy way out, so it feels like the best path. But, in the long run, it’s a trap. The house odds are against you and eventually, it will all come crumbling down.

Both experience and research are telling us that experimentation, where constant changing and measuring gives the team guidance and insight, is the approach that leads to long-term success and better designs.

That’s my recommendation. I’m sticking with it.

Jared Spool

Jared spends his days researching how teams create great designs as part of the team at User Interface Engineering. You can hear his latest research on how teams create their design principles at the upcoming UIE Web App Masters Tour across the US this spring. He is also behind the recently published masterpiece, Web Anatomy, even though his co-author, Robert Hoekman, Jr., deserves all the credit for the good bits. Make sure you follow him on the Twitters—he's pretty funny.

43 comments on this article

  1. *Playing* Meghan Ede’s “Bet Your Salary” = a great method for getting designers to really think hard about the possible “bottom line” effects of even the slightest “experimental” changes.

    This will do wonders for the “hey look at me… listen to what I can cook up and spew from my special mouth” types we all encounter/become sometimes (pointing at myself, because we all do it).

    Thank you for all you contribute Jared.

  2. I’m not sure that a years salary is a realistic bar to measure by.
    This approach can actually stump great ideas from coming to the table because the individual may cower away from the risk of a year’s salary.
    I can’t see a PM in their right mind taking someone’s salary either. If a project fails, stakeholders and BAs will most likely look to the PM for answers, I can’t see a PM simply passing on the blame to someone in her team while walking away with that team member’s salary and receiving no ownership for the problem.
    However, there is definitely something to be said for the concept- “Put your money where your mouth is”.
    A more realistic challenge would be to bet your bonus, vacation pay, or expenses. This makes it more realistic for the challenger and for others who support the challenger’s ideas. If someone agrees with the challenger’s recommendation, let them bet their bonus as well.
    I’m not entirely convinced that betting money on a project in the first place is the best approach to solving half-hearted recommendations, but I am in 100% agreement with the idea of making people “accountable” (pardon the pun) for their actions.

  3. How do you square this view with observations like those in this article:

    http://www.cennydd.co.uk/2009/statistical-significance-other-ab-test-pitfalls/

    “Cherry-picking only those design elements that are “proven” by an A/B test can be a route to fragmented, incoherent design. It may earn marginally more money in the short term, but it becomes hard to avoid a descent into poor UX and the long-term harm this causes.”

  4. Lee Sai Fon on

    Encouraging (or recommending!) an experiment is still a recommendation. Arguably it just has less commitment or weight behind it! The difficulty is that a significant number of companies actually want recommendations, not experiments. So, you would have to change company culture first. And if you don’t provide recommendations, you would probably be out on your ear anyway. And then who would pay the mortgage?

  5. I completely agree that user testing should be the backbone of making design decisions. However, once a pattern has proved itself to be superior through rigorous user testing, surely a UX designer can confidently recommend that solution.

    Surely there are “laws of nature” which are tried and true, having stood the test of time. To ignore these basic principles would be to squander time and resources to no more than prove that gravity does in fact exist.

    But with this caveat, the article is a great reminder that design decisions should be based on the user’s behaviour and not simply our guesses about the user’s behaviour.

  6. Pingback: Experiment don’t Recommend « Frictionless Design

  7. Kurt on

    Great post. Performance-driven, research-based UX improvements are the way to go. The funny thing is – experimentation doesn’t remove the need to make recommendations. You’re just recommending what to test. Which in many ways forces you to bet your salary every time (a good thing).

    If you routinely put up variants that DON’T out perform the problem, then you KNOW you make shitty recommendations. Maybe UX professionals should have to carry around their A/B perfomance test scores like baseball players do their RBI’s and batting average :-) .

  8. Jared your article made me smile and nearly created a slight panic as I was just wrapping up a report with a “Recommendation” page and wondering if I should change it to “Insight”. On a more serious note however I have noted a general trend that the vocabulary of design and Ux morphs every few years to accomodate and re-define our roles. That’s all good. The new word is indeed “insights” as receommendation are often to closed and inflexible. But doesn’t that bring us a bit backward in terms of pliability of how we convey our findings? I am on the fence myself and tend to give insights and recommendations when I am 100% sure. With insights I find it definitely involves the collective input of all stakeholders but then I worry that it is a weaker stance than a recommendation. I would like to hear thoughts on how others have found the use of the two ideas in presentation of findings.

  9. This is why I think it’s silly that design research and design are ever done by different people. Or that strategy and implementation are done by different people.

    This is why I run a design company – we don’t make recommendations, we make plans and execute them.

  10. Pingback: TwittLink - Your headlines on Twitter

  11. Lee Sai Fon on

    Christopher, you make recommendations, alright. You’re recommending your final design above another solution that you – or anyone else – could come up with.

  12. Jared – this sounds like an argument against consultants. Maybe I’m misreading, but the key (although subtle) difference in the approach has to do with the ability to iterate experimentally over time with a lower-stakes mindset; something that is not really possible when, as an outsider, you are paid to produce a concrete deliverable at the end of a discrete project.

  13. Josh Seiden on

    Jared,

    I wonder how you came to call this an experimental approach? Is *experimentation* actually the key? Or is it an indicator of a good process of iterative design refinement?

    My feeling on reading this is that “don’t make recommendations” was just a catchy way of saying, “be an involved designer and take ownership to ensure that your proposed solution actually works.” Which I think is what Chris was alluding to.

    I think that’s what you’re talking about too–based on the follow-up paragraph about discussing meaning. This is what a good design process does–starts with observation, goes to the deep implications, proposes a solution, then iterates until you have a satisfactory result.

  14. Pingback: Marriott rapid-prototypes a hotel lobby « Design and Innovation Daily

  15. I am glad to see that Jared finally changed his view on this.

  16. @Brett:

    I’m not sure that a years salary is a realistic bar to measure by.

    It’s more of a symbolic gesture than a specific amount. Sorta like when Apple announces that Steve Jobs is only get paid a salary of $1. The amount isn’t important — it’s the symbolism that makes people realize that recommendations cost something, especially poor ones.

    @Matthew:

    How do you square this view with observations like those in this article:

    http://www.cennydd.co.uk/2009/statistical-significance-other-ab-test-pitfalls/

    Excellent question. Cennydd makes excellent points about the problems that are found in experimentation, which is why the team needs to approach it carefully. But, just because experimentation is difficult doesn’t mean we shouldn’t go in that direction. After all, that’s what we’re being paid the big bucks for.

    @Lee Sai:

    Encouraging (or recommending!) an experiment is still a recommendation. Arguably it just has less commitment or weight behind it!

    Quite arguable. In our experience, when we’re working alongside the team, the discussion of experimentation and exploration carries far more commitment and weight than when someone is standing in the front of the room with a powerpoint decrying “Do this. Do that.”

    I will admit that many agencies will find this difficult and won’t go down this road. And many clients don’t want to be involved with their design. They just want the problem to go away. In either of those cases, in my opinion, bad design is the most likely outcome. If that’s acceptable, then keep doing things this way.

  17. @Kem:


    But doesn’t that bring us a bit backward in terms of pliability of how we convey our findings? I am on the fence myself and tend to give insights and recommendations when I am 100% sure. With insights I find it definitely involves the collective input of all stakeholders but then I worry that it is a weaker stance than a recommendation.

    I think the telling word in your question is “convey”. We want to move away from a model where, as the UX professional, we’re telling the team what they should know or do. We want to be facilitating a growing team knowledge, so that we’re all learning equally, yet from our own viewpoints.

    If you move away from conveyance toward collaboration, then the a “weaker stance” is no longer an issue.

    @Chris:

    This is why I think it’s silly that design research and design are ever done by different people. Or that strategy and implementation are done by different people.

    This is why I run a design company – we don’t make recommendations, we make plans and execute them.

    I agree completely. It’s all about collaborative teamwork. Even if the team just exists for the duration of the project work, with some members as permanent employees of the client and others from an agency.

    @Nick:

    Jared – this sounds like an argument against consultants. Maybe I’m misreading, but the key (although subtle) difference in the approach has to do with the ability to iterate experimentally over time with a lower-stakes mindset; something that is not really possible when, as an outsider, you are paid to produce a concrete deliverable at the end of a discrete project.

    It’s more of an argument against the type of consulting that is typical these days: “Hire us and we’ll tell you what to do.” I’m suggesting a move to “Hire us to work with you to help you learn how to make your design better.” This is our model at UIE and it’s been working great for many years. But it is a huge mindset change for those consultants doing things the other way.

    @Josh:

    I wonder how you came to call this an experimental approach? Is *experimentation* actually the key? Or is it an indicator of a good process of iterative design refinement?

    The real key is having the entire team working together through the iterations. You can have an iterative process that produces recommendations in the end, but that doesn’t work if key stakeholders weren’t part of the design thinking.

    So, yes, the end result will be a good iterative design refinement, but the difference is that everyone on the team is part of that.

    Keep those cards & letters coming.

  18. DJ on

    @Jared

    A great article as usual :)

    There’s lots of debate in practitioner circles about “who drives the bus” when it comes to practitioner teams. Always has been, perhaps always will be.

    I wonder what happens when a team operates in the way you describe.. What type of person & skillset is best to drive in this type of environment?

    Any tips?

    Thanks
    DJ

  19. Re: “However, some teams are using a different approach: “We’re seeing that our users have trouble with the field labels. We’d like to try an experiment and see if moving the labels to the top of each field makes an improvement.”

    I’d really like to take this approach with clients, but for most of them, getting them to do even the first iteration of usability tests can be a massive evangelical mission.

    A lot of early adopting companies take on usability people to tick the “buzzword box”, not necessarily take the activity seriously….let alone implement some of the more thorny fixes or do further tests :)

  20. There is almost always more than one way to solve a UI design problem. When testing isn’t possible a strategy we’ve used is to develop quick, specific visual sketches of all the design options we think are appropriate to solve a problem, and then discuss the pros and cons of each with the client team. They make the final (informed) decision and are invested in it as a result.

  21. Amy on

    What’s really interesting about this article is the meta-message.

    On one hand, nobody can reasonably argue that “stick it out there and hope” is a strategy for success. For all the reasons you lay out, and more.

    And yet, we can’t just go the other way. There are *real* downsides to a test-30-shades-of-blue mindset. It’s so terribly easy to split-test yourself right into a deceptive local maximum.

    If you take those two suckers & see what they’ve got in common, it’s easy to see that they come straight from fear. Or rather, coping mechanisms to DEAL with fear.

    Failure to engage the evolutionary process, choosing rather to declare from on high (and not suffer the consequences)? Comes from pretending that there can be Unquestionable Certainty.

    Relying on “cold, hard science” and excessive split-testing to prove the One True Shade of Blue? Comes from pretending that there can be certainty.

    If we want to do great work, we can’t get around the fact that, on this planet, nothing will ever be certain or easy — not in design, and certainly not in anything else.

  22. Cliff Tyllick on

    Jared, I agree with your position, but I wonder: Do you ever fire a client because they won’t live up to their end of the experimentation?

    And do you have tips the rest of us can use to overcome that resistance so our clients will become willing to learn with us?

  23. Pingback: Stop Telling People What To DoI | Ginny Shope Fowler

  24. Alexander on

    @Jared

    Let’s play “Bet Your Salary”: I’ll make a design recommendation, and bet my one year salary on it. We know what happens if i’m wrong, but what happens if i’m right?

    There’s no free insurance in this (design) world, so “go ahead: take out your checkbook. Write out a check for ‘my’ take-home salary, after taxes.”

    If i’m right about my recommendation, i’ll cash your check.

    Now i’m asking you: “How confident are you feeling about those recommendations?”

    Without reciprocity, this game is nothing but a simple threat based approach: if you are wrong i will punish you severely.

    My recommendation: make the bet game work both ways and recommendations will suddenly gain new insights.

    PS: the great article, made me think about some lost opportunities.

  25. Daemon on

    It comes with experience. After working in web, web design, UX, and interface design for over 10 years, I am quite certain that a recommendation that I give WILL IN FACT IMPROVE client’s website.

    Actually, when you think about it, every company, even freelancers DO MAKE A BET WITH THEIR WAGES every time they work. Because if they fail, they will not get hired again. So, want it, or not, every project you make is a bet, and the payoff of that bet is NEXT PROJECT.

    In total, yes, beginner designers should not make recommendations. Experienced veterans should.

  26. I would say that a redesign by a designer without usability knowledge is very dangerous. But a redesign by a designer with usability knowledge is going to outperform in 80% of the cases. When user study is made, then mistakes can be shrinked to 10%. The possiblity of making a mistake is always there, because user behaviour can be very unlogical when it comes to some unknown user groups.

  27. Jenna on

    I tend not to commit to things unless I’m 100% sure I can carry out that commitment and the same applies here. I’m not always 100% sure that changes I make will improve things so I make ‘suggestions’ and ‘recommend’ that they’re tested.

    However, you have to use a bit of common sense… obviously if a client suggests changing an online banking website to a bright pink flashing beast then I’m pretty confident I could ‘recommend’ against that and save the cost of testing.

  28. subsume on

    I’m not sure how useful the whole salary thought experiment is. I mean, its crafted to sound *really bold*. But boiled down its just silly and kind of a straw-man in that there is no pay-off if indeed the bet is hedged. Some designers might indeed make that bet.

  29. Hehe, reading this discussion I keep thinking about the obvious analogy to the stereotypical psychoanalyst response… something like:

    Client: Something’s wrong with my site, I need help!
    UX Consultant: Well what do *you* think is wrong with your site?
    Client: I don’t know, that’s why I hired you! Shouldn’t we do some testing or something?
    UX Consultant: Ok, let’s do some testing and see what happens.

    [post testing] Client: That was scary, we have a lot of problems, how should we fix them??
    UX Consultant: How do *you* think we should fix them?

    and so on…

    My clients would fire me if I didn’t come to them with ideas / recommendations for how to address problems identified in research. But we discuss the ideas and prioritize approaches together. We have a shared stake in the outcome. Maybe we’re already doing what you’re suggesting, but I don’t really see it as eschewing recommendations completely.

  30. Pingback: Weekly Roundup: #1 « Discovery Session… by Gerard Dolan

  31. Pingback: Weekly Roundup: Design Links Worth a Look #1 « Discovery Session… by Gerard Dolan

  32. Bernie Bunhole on

    Write them out a cheque from an account with less money in it. So it bounces and the get charged a dishonour fee.

  33. Pingback: 24 UX Articles to Start 2010 | UX Booth

  34. Traci Lepore on

    I wholeheartedly agree with you on this Jared. I think it takes a tactical and strategical way of approaching recommendations. Especially true when you are a consultant (like me). You want to build a good working relationship that is ongoing. You also want to try to get them to do what you believe is the “right” thing. And you aren’t really saying that you won’t tell them the same things, just think about how you say it first!

  35. Pingback: links for 2010-02-02 | AndySowards.com :: Professional Web Design, Development, Programming Freelancer, Hacks, Downloads, Math and being a Web 2.0 Hipster?

  36. Pingback: Interact Seattle » Blog Archive » User eXperience Digest #13

  37. Pingback: Yes, make design recommendations « Learning Game Design

  38. Pingback: Johnny Holland – It's all about interaction » Blog Archive » What Happens When You’re Gone?

  39. Pingback: Design e approccio consulenziale (reprise) - Alberto Mucignat

  40. Pingback: Buzz by eryk orłowski from Google Reader

  41. Pingback: Stop rekomendacjom – czy dawać klientom, czego chcą?

  42. Sorry but I don’t believe it is practical to follow this approach dogmatically. I don’t believe you are advocating the absurd end of the spectrum, ie the “30 shades of blue” someone mentioned but nonetheless I have not worked anywhere where testing every design decision is even remotely feasible.

    I would be fired if I refused to provide experience-based advice and opinion and instead kept deferring everything to “Let’s do some testing”. Of course, I disclaim everything and recommend we test … and in many cases get the “We don’t have time for that – just tell us what to do”.

  43. Pingback: Links for February 21st | jonathan stegall: creative tension