Why is it that when people think of involving a community in design their minds immediately turn to surveys and polls? Enough already. Involving a community in your design process doesn’t mean making the community make the design decisions… this is why we’re dreaming up new ways of engaging a community, qualitatively, in the design process for the Drupal 7 User Experience Project.
What’s so bad about a poll?
I think there are two key reasons why voting should generally be avoided in community design practice:
1. Your community is (probably not) made up of designers
what colour would you like? Which icons do you prefer? Do you want your navigation here, here or here? Which of these three visual design treatments do you like best?
When you’re out doing design research in person, you don’t ask people where they’d prefer their navigation. Rather you use conversation and observation to gain insight into who your users are, what they need to do with and any other information you need in order to make good design decisions. Why should it be any different online?
Humans, at the best of times, are pretty bad at telling you how they’re going to use things and what configuration will best suit them. We know that, don’t we? So what makes us think a poll is a good idea? A nd what do you, as a designer, do with data that says that 67% of the people surveyed prefer green? You don’t know anything about *why* these people like green, or in what context they actually hate green. This data contains no real insight.
2. Voting is like fast food cheap
It is easy and initially feels good, but it leaves you wanting more
A vote is super quick and easy to set up and similarly it is quick and easy to participate – you can sit back and watch the numbers roll in. ‘Look!’ you can shout, ‘see how many people are involved in our project!’
It’s easy to be seduced by the numbers. But ultimately a few quality interactions with your community are worth more than 1.000 empty gestures. Just as when you compare numbers in a qualitative study and a quantitive study, it’s not always a matter of more is best. Ultimately everyone is left wanting more. The designer wants more insight and the participant more engagement.
We need to be more thoughtful, more creative and willing to work a little harder in designing ways for communities to engage in an open design process. With that in mind, here’s a few activities we’ve been trying out on the D7UX project.
Extracting Existing Community Knowledge
This is an exercise that we first did on the Drupal.org project and are now running again for D7UX. Essentially we invite people to draw up a wireframe (in whatever medium they prefer) of a part of Drupal they think needs improvement. This time around we also asked people to make a quick screencast walk through to give us some more context and understanding of both the problem and the solution. Participants post their work either to their own blogs and give us a link or to the Flickr and Youtube groups we have for the project.
There is a reasonable amount of effort involved in participating in this activity, so we tend not to be flooded with responses. This is good, for a number of reasons:
- The data is quite tough to analyse.
- The people who do participate really do care about the project and the problem and have given it some real consideration. Each submitted artifact then becomes a discussion point for the rest of the community to gather around.
Working with a developer-led community like Drupal we do tend to get our ‘wireframes’ in pretty high fidelity. In fact, a wireframe for us is everything from a pencil sketch to a live implementation. Especially on a project like D7UX where there is a ‘back story’ to everything and almost every idea has already been considered and discussed at length. It is a great way to draw some of these conversations out earlier in the project than might otherwise be the case.
Crowdsourcing Usability Testing
Capturing Scale, Sharing Our Process
Continuing in the crowdsourcing vein, we have also been trying to capture the scale and diversity of the Drupal community by recruiting willing participants to help us do some user research on the project. We first attempted this exercise during the Drupal.org project but it didn’t really take off. Despite lots of interest, no actual tests emerged. So, it seemed like a good idea but we needed to tweak the process a little to make it more accessible and easier to engage. So far we’ve run one round of CrowdTesting for D7UX and happily, we’ve had about a dozen people around the world participate. And the information we’ve been collecting has been put to great use already.
What did we do differently this time? Rather than just putting the concept out there and saying ‘go for it!’, we have scheduled a series of dates throughout the project when testing will be undertaken. I’ve also been a lot more prescriptive about what needs to be tested and how to test it this time around. I provided a downloadable document with the materials (we’re testing paper prototypes at the moment) and a script to follow, plus some interviewing tips for newbies.
Participants conduct a short test and post the results to an online questionnaire. We also encourage participants to video their test and post the video to our Youtube group (or elsewhere if they prefer). The posted video is particularly helpful in allowing us to analyse and contextualise reported findings.
What are the benefits of this activity? Not only do we gather more research data from all around the world (and yes, we are still coming to terms with the challenge of languages other than English), but we are also making user research much more accessible to people who might otherwise never had exposure to it. We are really hoping to continue to promote the practice of user observation throughout this project. And we hope to see real user observations used as evidence as we work with the community to debate the best approaches to various aspects of the interface.
‘Pimp Your Admin’
Extracting Existing Community Knowledge
When we first started on the D7UX project, we had a feeling that lots of people out there were ‘hacking the interface’ of their Drupal admin to make it more usable for their clients. We knew this, of course, because we’d seen a few instances of it. It struck us that this would be a great way to do some ‘requirements gathering’: to take a look at as many ‘hacked interfaces’ as possible and see if there were similarities in the changes that people were making to the admin interface to make it more human friendly.
And so we came up with ‘Pimp Your Admin’ in which we invite people to walk us through the customisations they’ve made (or that they regularly make) to the admin interface. We launched this at a conference, Drupalcon DC, where we sat down with people and did the screen recording and interviews in person. But this is also an online activity and members of the community from around the world have since prepared screencasts sharing their own personal interface modifications. This has been interesting both for us and for the community – you don’t really get to see other people’s admin interfaces very often!
As we suspected, there are definite ‘themes’ in the changes made to the interface and even at this early stage we know that several of these will be mirrored in our proposed design. We’re continuing to capture this material throughout the project too.
Having the opportunity to work within this community context is an amazing challenge and opportunity, we’re really trying hard to be as thoughtful about the way we engage with the community as we are with the design work we produce but it is a constantly evolving process of trial and error and refinements.
Want to see where we’ve gotten with it so far? Be sure to check out our recently released Initial Concepts and Directions and be sure to let me know what you think!
Top image by yoroy