Here in Drupal7 User Experience Project land we’ve been moving from ‘iteration zero’ to the actual production iterations. In iteration zero we’ve been doing a lot of our strategic thinking and documenting, but now it is time to start producing output that the developers who are working with us on this project can turn into something that will be contributed to the Drupal7 Project.

There is a real cadence to the project, and although there is no time in the schedule for us to take a breather, between you and I, it has been impossible for us not to do so (just a little), before heading back into the fray. I’ve noticed this effect a few times in agile projects and I think that I’m going to try to encourage project managers to allow for a little breather at this point in future projects I work on.

I thought I’d take a moment to share with you some of the other shifts that start to happen as we move from Iteration Zero into the Production Iterations.

Communication Framework: From Abstract to Concrete

As I’ve mentioned in the past, a big part of the time we spend on this project is spent either communicating with the community about the work we’re doing, our process and our ideas, or trying to work out a better way to communicate with the community.

One of the biggest challenges of the Iteration Zero stage in the project is that it is, by and large, a series of quite abstract and strategic discussions.

It is really easy to forget that many people find abstract and strategic discussions really difficult. I think there are particular types of brains that embrace the abstract better than others, but experience in this project phase is also very helpful.

In Iteration Zero there is often a lot of writing and talking and not much making/showing – this can create a very challenging environment for project participants. It is pretty easy for people to have vastly different interpretations of the same concept and it can be difficult to make sure that everyone is on the same page with the higher level strategy for the design and product. I’ve experienced this recently not only with the Drupal project, but with a few other projects I’m involved in.

Abstract discussions can be difficult to grok due to their predominantly conceptual nature.

It is pretty easy for people to have vastly different interpretations of the same concept and it can be difficult to make sure that everyone is on the same page with the higher level strategy for the design and product.

I can’t begin to tell you how many times I have explained and re-explained the very same concept, each time thinking that it’s been communicated clearly, only to discover that we still have at least two very different ideas about how something is going to work. There are at least two reasons for this: firstly I have to take responsibility for communicating – if the message isn’t being received I have to re-evaluate either what or how I’m communicating. We also have a second and somewhat unique problem when communicating with the Drupal community and that is that they have a tremendously strong mental model of How Things Work In Drupal. Every time an idea is presented the community almost invariably tries to map it directly to their mental model of How Things Work In Drupal – this is natural and what we *do* with mental models, but when the concept we’re suggesting actually breaks the model, we can run into trouble. It just doesn’t compute! It becomes abstract, difficult to understand, as we have to try to find ways to make concepts more concrete.

Iteration Zero can be a stressful time as a result of this abstraction – people aren’t really certain that they know what you’re talking about, but you’re also asking them to make decisions that will be really significant in shaping the product they’ll be getting at the end of the project.

I think it’s pretty common for people to be fairly fraught towards the end of Iteration Zero.

Thank goodness it is also around this time that something excellent happens – things start to become a little more concrete. There are still a bunch of abstract concepts that need to be agreed on, but as designers we’re also starting to get our heads around exactly how things will fit together and we can start to communicate that.

This is around about the time that we had a fundamental overhaul of the way we’ve been communicating with the Drupal community and interested onlookers on this project.

Towards the end of Iteration Zero we were starting to get a little down about the some of the feedback we were getting on the D7UX project – people were saying that they didn’t want to get involved because it was too intimidating for people who didn’t have UX experience and expertise, that they didn’t think it would actually happen or be a success, that they felt that the discussion was too disjointed and widespread.

It was clear to us that we needed to change the way we were engaging with the community to help them help us. Essentially, we needed to change the structure of the conversation from it’s abstract Iteration Zero format to a more concrete format appropriate to the production iterations, and, we suspected, to a format that most of our participants would find much more comfortable.

Over the course of a day, we created a ‘Project Framework’ on the D7UX site by breaking down the project into it’s main component parts and providing a wireframe, description and outline of ‘what we’re thinking’ for each part. Threaded comments allow people to give their thoughts on each component as it evolves over time.

Allowing people to participate in a place that is most comfortable to them is a key part of our communications strategy. We wanted to continue with this strategy even as we move into this new phase of the project, but also to aggregate the discussions into one place and to facilitate this we created a system of tags for the project components and put together a series of Yahoo Pipes to pull tagged content together. We added a link to these pipes on each of the component pages in the framework.

It was a pretty big overhaul and quite a time consuming process, but almost immediately we noticed a significant difference in the way that people were communicating with us on the project – the interactions became much more focussed and productive and felt a whole lot more positive, and that trend seems to have continued. It also makes it much easier for us to be more conversational with the community in the project – thanks to the simple addition of threaded comments and also the aggregation of the main part of the conversation into one place.

Overall, we’re really pleased with the effect that changing the format of the conversation from abstract to concrete has had on the project to date and the effort involved has already been rewarded.

The Challenge of System Design with User Stories

Another major challenge that we’re butting up against at the moment is to try to make a system design fit into an agile environment.

I’m a big fan of agile methodologies and have had a long term interest in finding better ways for UX practitioners to engage in agile methods. Unfortunately, there is no denying that pushing a design project like this one into agile iterations is tricky.

The way that our user stories are being developed at the moment is that the project manager from the developer agency (Acquia) is writing user stories then pushing them over to us to check that they are right and for us to adjust and re-order as required. To date, we have mostly let them sit in a large spreadsheet whilst we focus on the design strategy (iteration zero) and try to ignore the need for user stories.

We’ve done quite a bit of work on developing an Audience Matrix that allows us to take quite sophisticated ‘views’ of the design from multiple audience perspectives, but to translate this into user stories is untenably complex. The alternative to date has been overly simplistic. We are struggling to find a way to make good use of our audience modeling work to date without breaking agile.

Another issue that we’re butting up against is the nature of system design and templating in an agile environment. There are sets of design elements or template components that would ideally be designed in components then re-used throughout the project – for this project examples of these would be the admin header, the overlay window and the edit-in-place interaction model. Describing these using user stories is incredibly clumsy and inappropriate.

Once these elements are built and we start looking at user pathways that make use of them for particular tasks and outcomes then user stories will come into their own, but it seems that in the same way that developers need a piece of time to set up their development environments and databases without requiring user stories being used, designers need some time to get the ‘design environment’ set up without requiring user stories.

Again, this is something that I’ve come across on a number of agile project I’ve worked on but I’ve not seen any allowance for this way of working in Agile UX project methodologies.

If you’ve had similar challenges and some ideas or solutions then I’ve love to hear from you!

Update on Crowdsourcing Usability Testing

In my last update I was telling you about the Crowdsourcing Usability effort we had launched. Since then we’ve seen that WordPress have launched a similar campaign and they managed to get coverage in the New York Times no less, so we will be watching their progress with interest. Exciting times!

Launching Microprojects

Want to dip your toe in Open Source Design? Help out with D7UX? Well, here’s a great way to give it a try – sign up to help out with one of our microprojects! You need to commit just 12 hours over 3 weeks, but you’ll get a feel for what it’s like to design with one of the most vibrant and clever communities you could ever come across. Be warned, it’s challenging but potentially very addictive!

Why is it that when people think of involving a community in design their minds immediately turn to surveys and polls? Enough already. Involving a community in your design process doesn’t mean making the community make the design decisions… this is why we’re dreaming up new ways of engaging a community, qualitatively, in the design process for the Drupal 7 User Experience Project.

What’s so bad about a poll?

I think there are two key reasons why voting should generally be avoided in community design practice:

1. Your community is (probably not) made up of designers

what colour would you like? Which icons do you prefer? Do you want your navigation here, here or here? Which of these three visual design treatments do you like best?

When you’re out doing design research in person, you don’t ask people where they’d prefer their navigation. Rather you use conversation and observation to gain insight into who your users are, what they need to do with and any other information you need in order to make good design decisions. Why should it be any different online?

Humans, at the best of times, are pretty bad at telling you how they’re going to use things and what configuration will best suit them. We know that, don’t we? So what makes us think a poll is a good idea? A
nd what do you, as a designer, do with data that says that 67% of the people surveyed prefer green? You don’t know anything about *why* these people like green, or in what context they actually hate green. This data contains no real insight.

2. Voting is like fast food
cheap

It is easy and initially feels good, but it leaves you wanting more

A
 vote is super quick and easy to set up and similarly it is quick and easy to participate – you can sit back and watch the numbers roll in. ‘Look!’ you can shout, ‘see how many people are involved in our project!’

It’s easy to be seduced by the numbers. But ultimately a few quality interactions with your community are worth more than 1.000 empty gestures. Just as when you compare numbers in a qualitative study and a quantitive study, it’s not always a matter of more is best. Ultimately everyone is left wanting more. The designer wants more insight and the participant more engagement.

We need to be more thoughtful, more creative and willing to work a little harder in designing ways for communities to engage in an open design process. With that in mind, here’s a few activities we’ve been trying out on the D7UX project.

Crowdsourcing Wireframes

Extracting Existing Community Knowledge

This is an exercise that we first did on the Drupal.org project and are now running again for D7UX. Essentially we invite people to draw up a wireframe (in whatever medium they prefer) of a part of Drupal they think needs improvement. This time around we also asked people to make a quick screencast walk through to give us some more context and understanding of both the problem and the solution. Participants post their work either to their own blogs and give us a link or to the Flickr and Youtube groups we have for the project.

There is a reasonable amount of effort involved in participating in this activity, so we tend not to be flooded with responses. This is good, for a number of reasons:

• The data is quite tough to analyse.
• The people who do participate really do care about the project and the problem and have given it some real consideration. Each submitted artifact then becomes a discussion point for the rest of the community to gather around.

Working with a developer-led community like Drupal we do tend to get our ‘wireframes’ in pretty high fidelity. In fact, a wireframe for us is everything from a pencil sketch to a live implementation. Especially on a project like D7UX where there is a ‘back story’ to everything and almost every idea has already been considered and discussed at length. It is a great way to draw some of these conversations out earlier in the project than might otherwise be the case.

Crowdsourcing Usability Testing

Capturing Scale, Sharing Our Process

Continuing in the crowdsourcing vein, we have also been trying to capture the scale and diversity of the Drupal community by recruiting willing participants to help us do some user research on the project. We first attempted this exercise during the Drupal.org project but it didn’t really take off. Despite lots of interest, no actual tests emerged. So, it seemed like a good idea but we needed to tweak the process a little to make it more accessible and easier to engage. So far we’ve run one round of CrowdTesting for D7UX and happily, we’ve had about a dozen people around the world participate. And the information we’ve been collecting has been put to great use already.

What did we do differently this time? Rather than just putting the concept out there and saying ‘go for it!’, we have scheduled a series of dates throughout the project when testing will be undertaken. I’ve also been a lot more prescriptive about what needs to be tested and how to test it this time around. I provided a downloadable document with the materials (we’re testing paper prototypes at the moment) and a script to follow, plus some interviewing tips for newbies.

Participants conduct a short test and post the results to an online questionnaire. We also encourage participants to video their test and post the video to our Youtube group (or elsewhere if they prefer). The posted video is particularly helpful in allowing us to analyse and contextualise reported findings.

What are the benefits of this activity? Not only do we gather more research data from all around the world (and yes, we are still coming to terms with the challenge of languages other than English), but we are also making user research much more accessible to people who might otherwise never had exposure to it. We are really hoping to continue to promote the practice of user observation throughout this project. And we hope to see real user observations used as evidence as we work with the community to debate the best approaches to various aspects of the interface.

‘Pimp Your Admin’

Extracting Existing Community Knowledge

When we first started on the D7UX project, we had a feeling that lots of people out there were ‘hacking the interface’ of their Drupal admin to make it more usable for their clients. We knew this, of course, because we’d seen a few instances of it. It struck us that this would be a great way to do some ‘requirements gathering’: to take a look at as many ‘hacked interfaces’ as possible and see if there were similarities in the changes that people were making to the admin interface to make it more human friendly.

And so we came up with ‘Pimp Your Admin’ in which we invite people to walk us through the customisations they’ve made (or that they regularly make) to the admin interface. We launched this at a conference, Drupalcon DC, where we sat down with people and did the screen recording and interviews in person. But this is also an online activity and members of the community from around the world have since prepared screencasts sharing their own personal interface modifications. This has been interesting both for us and for the community – you don’t really get to see other people’s admin interfaces very often!

As we suspected, there are definite ‘themes’ in the changes made to the interface and even at this early stage we know that several of these will be mirrored in our proposed design. We’re continuing to capture this material throughout the project too.

Having the opportunity to work within this community context is an amazing challenge and opportunity, we’re really trying hard to be as thoughtful about the way we engage with the community as we are with the design work we produce but it is a constantly evolving process of trial and error and refinements.

Want to see where we’ve gotten with it so far? Be sure to check out our recently released Initial Concepts and Directions and be sure to let me know what you think!

Top image by yoroy