In this round-up of book reviews we are trying to broaden your perspective a little bit. We review the typical UX books, like ‘Beyond the Usability Lab’ and ‘User Experience Re-Mastered’, but we also try to get inspiration from ‘Innovators’ and ‘Layout Essentials’.

Beyond the Usability Lab

Type: theory
Authors: Bill Albert, Donna Tedesco, Thomas Tullis
Publishers: Morgan Kaufman
ISBN: 978-0123748928
Details: 328 pages, paperback

When you are new to the business and need a hands-on tour through the UX process.

Online usability testing seems to have become ‘the new kid on the block’. In the last few months we’ve received four books from different publishers that all covered this topic. Is it really a topic so complex that it needs several books explaining you how to do it?

Of course you can start directly with online usability testing, gathering valuable information about your users and product. There are enough easy tools to help you along the way (e.g. Loop 11 and Usabilla). But this book does provide some valuable background and guidelines to help you get the best out of your usability tests. It is a very complete guide, containing a lot of valuable information, even more basic information and probably a bunch of information that will never be useful for you.

The book is very clearly written and provides a lot of examples.

Here is an overview of the topics

• An introduction to online usability testing
• Planning the study
• Designing the study
• Piloting and launching the study
• Data preparation
• Data analysis and presentation
• Building your online study using commercial tools
• Discount approaches to building an online study
• Case studies
• Ten Keys To Success

User Experience Re-Mastered

Type: theory
Authors: Chauncey Wilson (editor)
Publishers: Morgan Kaufman
ISBN: 0123751144
Details: 396 pages, paperback

When you are new to the business and need a hands-on tour through the UX process.

In the last few years loads of real specialized UX books have been released. All of them contain hundreds of pages focusing on one specific subject, ranging from sketching to creating personas. For most designers these books contain too much information. For them ‘User Experience Re-mastered’ might be a great alternative. This book gives an overview of the design process, including practical examples. But it doesn’t do this by rewriting what has already been said in other books; it is a collection of chapters from existing books. I really like this approach, because there is already so much great stuff around… why do it again?

The content of the book is great for students and people who are new to the field, but there isn’t much new under the sun. One disadvantage of re-using chapters from exisiting books is that some examples and images do seem to be a bit old. There are even some pictures that are bad quality, which is a shame. But don’t let this scare you away, there are still some really interesting chapters such as ‘User Needs Analysis’, ‘Persona Conception and Gestation’ and Bill Buxton’s ‘Sketching: A Key to Good Design’.

Innovators: Shaping Our Creative Future

Type: inspiration
Authors: multiple
Publishers: BIS Publishers
ISBN: 978-94-9022-801-9
Details: 408 pages, hard cover

Be inspired by the innovators that are shaping our future.

There were high expectations when we received a copy of this book; it isn’t every day that a book tries to give an overview of the innovators of our creative future. Did it succeed? The fact that multiple friends already ordered their own copy might give an answer to that.

The idea behind the book was to let “international experts from twelve fields of creative practice, from fine art and photography to graphic design, architecture and ecology [...] each select five people who are making significant contributions to the way we experience life and visual culture.” This resulted in a beautiful book with 60 innovators and their work. Each innovator is introduced by a short text with a lot of graphic. The book screams to be looked.

While paging through this book you can enjoy many brilliant ideas. It is a nice way to get energy for your own projects. I personally liked the broadness of the fields, simply because the most unexpected fields seem to inspire me. My personal favorite innovators in the book were web artist Yogo Nakamura, rock band Radiohead and designer Viktor & Rolf. What are yours?

Layout Essentials: 100 Design Principles For Using Grids

Type: practical
Authors: Beth Tondreau
Publishers: BIS Publishers
ISBN: 978-90-6369-211-7
Details: 208 pages, hard cover

Learn to follow the rules and you’ll be able to gain more creative freedom.

As designers we are always struggling to create unique but perfect designs. It takes some time for us to acknowledge that behind each brilliant design isn’t just our bright mind, but also a world of existing rules. Among these rules the one about applying good grids must be one of the most important ones. Almost all good design around you uses a specific grid. Understanding the power of grids can help you become a much better designer, and that’s exactly where this book tries to come in. At it’s core the book tries to focus on graphic designers, but I believe that also interaction designers could benefit greatly from this knowledge. When we understand the power of grids we can understand why certain sources of information (websites, UI, …) are more succesfull than others.

At the start of the book she shortly introduces the vocabulary for grids (columns, modules, margins, modular, multicolumn, ..), but right after that she jumps into the 100 examples. The book itself is beautifully made and the full color plates are really inspiring. Each example shows you the helpfulness of grids without having to lose graphical power and freedom. I really liked this practical approach of showing examples with a bit of theory., but I can imagine that a lot of people like to have more background information before they start applying the grids. So if you love theory: don’t buy this book. If you love the more practical approach: get it.

PS: if you want to directly start using grids for websites, go to the 960 Grid System site.

Buy UX Books

The disciplines of interaction design and architecture share a number of common traits—such as a focus on solving problems for people and encouraging people to interact with products and environments in new and exciting ways—and each discipline can learn much from the other.

These eight videos highlight the work of people who see and celebrate the connections between interaction design and architecture.

The SENSEable City

Carlo Ratti, director of the SENSEable City Lab at MIT, presented this talk at Lift Conference 2009.
Original video

Stewart Brand on the Long Now

Stewart Brand, who is working with computer scientist Danny Hillis to build a 10,000-year timepiece called the Clock of the Long Now, presented this TED talk in 2004.

New Soft City

Dan Hill, a senior consultant at Arup in Sydney, presented this keynote at Interaction ’10. Hill’s presentation was too beautifully described to paraphrase it.

Muti-sensory interaction design now merges with architecture, planning and an urbanism informed by a gentle ambient drizzle of everyday data – and so a new soft city is being created, alive once again to the touch of its citizens.

Changing Things: The Internet of Things is not what you think it is!

Usman Haque, director Haque Design + Research Ltd, founder of Pachube.com and CEO of Connected Environments Ltd., presented this talk at Lift Conference 2009.

The City as an Interaction Platform

In this panel discussion from PICNIC ’09, Ben Cerveny, Greg Skibiski, Adam Greenfield, Beth Coleman and Atau Tanaka discuss how the technology of today and the near future will transform the experience of living in cities.

The Long Here, the Big Now, and Other Tales of the Networked City

Adam Greenfield, head of design direction for Nokia and author of Everyware, presented this talk at PICNIC ’08.

Future urban life will thrive on new modes of perception and experience, based on real-time data and feedback. What will the networked city feel like to its users? How will it transform our sense of the metropolitan?
-Adam Greenfield

Cameron Sinclair on open-source architecture

Accepting his 2006 TED Prize, Cameron Sinclair demonstrates how passionate designers and architects can respond to world housing crises. He unveils his TED Prize wish for a network to improve global living standards through collaborative design.

Architecture that repairs itself?

In this 2009 TEDGlobal, Rachel Armstrong says we need to outgrow architecture made of inert materials; instead, she proposes a not-quite-alive material that repairs itself.

For more videos on UX, don’t forget to check out Johnny TV

———
Header image taken from Dan Hill talk

With a mobile phone, you can watch movies/videos/TV programs online anywhere at almost any time. That ubiquity has brought tremendous convenience to our lives, but on the other hand, the value of them has been diminishing. We don’t very much/at all care about where we can get a clip or if we miss a drama episode. What if, however, we have a movie that can be watched only when certain conditions are met?

This concept is partly like a concept I’ve already introduced in my blog, but a little different.
OK, so you hold this device in front of you and watch a clip, but the clip can be played only if you are at a particular place. You have to BE there to see the whole movie.

For example, you can see

1. a movie’s secret story at an actual scene
2. a sequel clip to a TV commercial with how to apply for an exclusive present announced at the end
3. your favorite actress choose clothes at a boutique
4. a comedian introduce which spots you should visit in a touristy place

etc. etc.

Or you can shoot a video of yourself for your boyfriend saying “Hey, how long do you think I’ve been waiting for you? I’m going home.” so that he can see it when he finally arrives.

By limiting the geographical conditions for viewing movies this way, you might expect audiences to visit places you like.

It would be so much fun if videos can be seen in 3D from any angle (no, not from below).

How can users quickly create a timed transcript of any video on the web? That’s Mozilla’s latest design challenge, in collaboration with the Participatory Culture Foundation, challenges teams to design an intuitive interface for creating and improving subtitles for any video on the web. In this article I’ll share some ideas on how to interpret usability testing results like those presented by the Mozilla Labs team.

Mozilla Labs Design Challenge?

The Mozilla Labs Design Challenges are events for interaction designers around the world to stimulate innovation in user interface design. In three months (March 9th to June 18th) this 7th design challenge will guide participants through a full design cycle:  ideate, select, prototype, evaluate.

The Design Loop

Mozilla Labs kicked off with an evaluation of the Participatory Culture Foundation’s current subtitling tool. Design teams can test-drive the subtitling tool and an interesting part of the briefing is a usability study of four first time users.

Meet John, Jane, Cynthia, and George

Mozilla Labs conducted a usability test with four typical users who had never seen the subtitling tool before. These users performed basic tasks in the subtitling tool: transcribe, sync, and review subtitles. The tests were conducted with an external testing service. On the website of the design challenge you’ll find screen recordings and some general observations for each of the four individual sessions.
John (20), Jane (64), Cynthia (52), and George (28) were asked to use PCF’s current subtitling tool and think aloud to share their experiences. After the test they gave short written response. Based on these responses and user behavior, four pages of summarized observations were generated. But, how can we translate these observations into ideas?

Categorize usability challenges

Creating a subtitle with the current tool is done in three steps: transcribe, sync, and review. Participants worked their way through these steps and encountered challenges on their way. We labeled these challenges with one (or more) of the three steps or as a general problem. While categorizing the usability issues into different tasks,  we assigned a severity rating for each issue. Our goal was to keep the ratings simple, for example: critical [3], serious [2], minor [1], and no issue [0].
Once everything was categorized, numbered, and rated, we created an overview of all the issues. Duplicates were combined into a single issue in order to clean things up (example C1).

Visualizing

Sketching over wireframes

We’re not writing a book – we’re designing a graphical user interface. The four participants were asked to verbalize their experiences. Our participants processed their observations and we captured them in spoken and written form. The Mozilla Labs team did the same and summarized their observations in about 10 bullets for each session. Somewhere in the design cycle we need to switch modality: from text only to a combination of image and text. The earlier, the better.
Visualizing interaction challenges in your interface helps to kickstart your ideation process.

User feedback with Usabilla

Tips for visualising test results

• Create your own toolbox to collect information about your users and the problems they encounter. Using different tools to measure user behavior and collect feedback helps you to get a better picture of your users and can be extremely useful in the ideation process.
• Visual deliverables bring your test results to life. Use visual feedback in your deliverables to pinpoint the most important problems and to share your observations with your team.
• Categorize usability issues (transcribe, sync, review, general) when you visualize your test data. You can use the categorized overview of issues as legenda and label your sketches with the numbers of each issue.
• Fast is good. Try to keep your evaluation cycle as agile as possible. Lean and mean tests don’t necessarily slow down the design process. Take small steps and verify your choices with quick tests.

Entries are open for the Mozilla Collaborative Subtitling Challenge from now until the 26th of April. For more information, see the Mozilla Challenge site.

A strategy is a set of coordinated, orchestrated, planned actions, or tactics, which will take you along a journey to reach a desired future state, over an established period of time. Design objectives are conditions or outcomes that a project must meet, often of tactical nature. User experience (UX) strategy shouldn’t therefore be confused with design objectives. This article is about how to plan and coordinate actions to organisationally achieve good UX.

The idea for this article was sparked by a cacophony of opinions on what constitutes UX strategy voiced recently in the twittersphere. Nothing currently posted in the blogosphere or in the UX community websites about UX strategy hit the right chord with me. I decided to add my two cents to the debate and try to bring a new perspective, based on my professional experience.

I think that the confusion stems partly from Jesse James Garrett’s famous, groundbreaking, commendable book, The Elements of the User Experience, where he employs the term UX strategy to name what was traditionally called a project’s design objectives or goals. Design objectives may consist of things like this:

• Reduce checkout drop-off rates to 30% from 70%
• Make it easy for new members to sign up for the web site
• Minimise the number of steps required to search, select and book a flight

A single project is always a tactical move; the use of the term “strategy”, in the context that Garrett uses it, sounds a bit off beat to me, and erroneous. It’s not my goal to flame JJG or anyone here – he’s an author who deserves my deep respect – but clear distinction between tactics and strategy is necessary.

So, what is UX strategy then?

User experience comprises everything a customer touches, hears, or sees from your organisation. From your products’ usefulness and functions, their form, performance, packaging, user guides, to the accompanying services like your ordering processes, customer support, billing, through social responsibility and environmental concerns, all of these aspects will form the customer’s appreciation of how they – consciously or not – experience your organisation (PDF).

UX strategy is therefore about the big picture. The ability to deliver a compelling, engaging and successful user experience is directly impacted by your organisation’s ability to orchestrate marketing, engineering, IT, product development, sales, and operational efforts. To that end, it must be approached from a higher vantage point – at the organisational level – not from a project level. In many politically charged, siloed organisations the orchestration required can represent an overwhelming endeavour and a daunting task. A well delineated strategy, however, will provide you the means to workaround those barriers. It will help you align all the ducks, break silos, build commitment and – why not? – gain consensus, and motivate the troops. The strategic plan will set the agenda for developments and actions to deliver on your strategic vision and objectives, usually in the 3 to 5 year timeframe.

UX strategy how-to

The process for defining your UX strategy is not very different to what you’d typically do to define your organisational overall strategy, corporate plan, or growth plan – whatever it’s called in your business. If your business doesn’t have a strategic plan yet, you should take care of that first. It generally comprises the following steps:

• Figure out where you are today
• Work out where you want to be in the future
• Choose and prioritise actions, while addressing any issues
• Map out the journey
• Get the job done

Strategic planning, like typical UX work, is a multi-disciplinary exercise. You’ll need a team of 5 to 8 people, usually representing a cross section of your organisation, i.e. people from marketing, IT, sales, product development, operations, financial, HR, etc. These must be people with power to make it happen, and willing to make a positive contribution to the planning process.

Where are you today?

This phase’s goal is to take a picture, a instant view of what’s happening now both internally and externally to your organisation. The reason is pretty straightforward: to go where you want to go, you need to know where you are first. Otherwise, which direction to take? Here’s what you need to look at:

The external landscape

If you haven’t done it yet, define your audiences. The most useful way to do this, in UX terms, is to develop a character set of personas representing your key audiences. Personas are excellent devices to communicate your audiences’ key characteristics, attributes, expectations and needs to your internal stakeholders and peers, and to build a common understanding of what needs to be put in practice to meet their requirements. Personas are also a very useful asset that can be reused for a myriad of other initiatives and, if you spend time tweaking and improving them over time, will have a long-lasting lifecycle.

You’ll also want to understand where you stand in the marketplace, review your positioning, benchmark your products and services, and compare them against the competition. The goal here is not to copy them, but to identify gaps in the offering and envision opportunities to better your own stuff. Pro tip: some people think that, because people out there are doing things in a certain way, this most certainly represents a best practice. Don’t fall in this trap! Focus on differentiation and innovation instead.

Lastly, analyse industry trends, paying careful attention to social change brought by new technologies, and don’t forget to look at your supply-chain and network of partners.

The internal landscape

Make an assessment of your organisation’s capabilities and competencies. You can do that through a Usability Maturity Model (UMM) assessment (I prefer User Experience Maturity Model instead, and will refer to it as UXMM heretofore). Don’t do that off the top of your head, though. Instead, employ solid assessment models and be rigorous and follow a protocol – as you do with usability testing, right? – because credible data and trustworthy, honest reporting will get you much needed stakeholder buy-in. The UXMM assessment is basically the crossing of UX skills, tools and processes on one hand, and a grading system, on the other hand. The resulting matrix will reveal where your organisation stands in the scale, and will highlight the gaps that need to be bridged. Regrettably, existing UXMM assessment frameworks can be difficult to use, and little literature or documentation is available on the subject. The Usability Professionals Association has a useful article (unfortunately for members only) comparing the different UXMM assement frameworks available today. Here’s an example of UX Maturity scale:

The User Experience Maturity Model

(Adapted from Bruce Temkin’s Experience-based differentiation maturity)

Map out your business processes and pay careful attention to handover points between channels, e.g. online to phone. If you work for the government, examine cross-agency baton-change points, e.g. commuters changing from buses to trains. Those are typically areas where organisational tectonic plates can meet, causing user experience havoc. Customer journey mapping (CJM) will help you address these kinds of issues.

You may also find it useful to develop a set of staff personas, and check the state of the systems and applications portfolio supporting staff work. Draw up a clear picture of your IT infrastructure and its organisation.

Le fric, c’est chic. Freak out!

UX is not about being nice to users; it’s about money. In effect, UX activities aim at enhancing productivity, improving conversion rates, and delivering higher ROI, to list a few, which ultimately translate into either reduced costs or higher income. Therefore, try to get a grasp on your UX-related expenditures, which may include marketing and IT budgets. The goal here is to identify opportunities for rationalisation, to align and synchronise investments where appropriate, to ensure equity across business units (thereby preventing asymmetric implementations between rich and poor units, for instance), and to deliver a higher ROI.

Finally, understand your stakeholder’s priorities, drivers, culture and values. Don’t forget to look at any regulations framing your activities.

All of the above may sound like a lot of work, but if you have already got a business strategic plan, you can piggyback on most of the information in there and concentrate on UX stuff like UMM, persona development, customer journey mapping, etc. Don’t bloat your head with detailed information and avoid at all costs a too lengthy data gathering processes. You’ll get stuck otherwise.

Where do you want to go?

Now, it’s time to make projections in the future. It’s not about wishful thinking or where you think you might be; it’s where you want to be.

Once the data gathered in the previous step is analysed, some obvious gaps will surface. It’s then time to find creative, innovative ways to bridge those gaps, and make informed choices.

Choosing and prioritising actions

One straightforward goal could be to gain, say, one full step of the UXMM ladder, or move up to the next step. Depending on where you are in the scale, moving up to the next step may require some years of carefully planned and timed actions.

Samsung cultural makeover, for example, took ten years of hard work and hefty investments – which included the establishment of an in-house design academy – to change from a status of me-too designer of cheap electronic products to winning awards in international design contests.

The outcomes of this phase are:

• the strategic vision
• mission statement
• key UX principles
• key objectives and areas of improvement

Developing UX principles is very important; they are the guiding stars to all of the great things that you are going to create for your customers. An example of UX principle could be “Involve users in every phase of the design process”.

Spelling out the above is crucial for the successful communication of the strategy, and to generate commitment, motivation and engagement within the troops. Make sure you address any issues and zap any roadblocks that may lie in your way.

Planning your journey

Start by collecting any low hanging fruit, now. How the plan is laid out may vary depending on your organisation’s needs, but it typically charts who does what when, and what needs to be in place when. You’ll also want to know when do you’ve reached your targets. Metrics are your friend here.

Getting the job done

Roll up your sleeves and charge ahead. Constantly monitor changing conditions along the way to make any necessary adjustments of course and don’t forget to celebrate when you hit your targets.

Final thoughts

The higher an organisation is up the UXMM ladder, the more the UX strategy becomes fully embedded in the overall strategic plan (as opposed to being a separate, standalone thing). Indeed, when UX is part of an organisation’s fabric – not treated separately – its strategic nature is core to the business, like it is the case at Amazon, Apple, IKEA and BMW.

UX practitioners and designers alike will benefit form talking the talk and walking the walk of their businesses and/or clients. You may find useful some readings on strategic planning, and I’d recommend the following:

• Simplified Strategic Planning, by Robert Bradford and J. Peter Duncan with Brian Tracy
• High Impact Tools and Activities for Strategic Planning, by Rod Napier, Clint Sidle and Patrick Sanaghan

In a new series from Johnny Holland, we turn the light onto the community to discuss hot topics and issues. Read on and discuss! We begin on communication.

You have spent time with your team crafting a UX vision for a product you are working on. You have spent a few weeks outlining who the key users are, the critical journeys, design principles, the product framework and who needs to work on what. There is genuine excitement in the workshop about what will be required to take the UX vision forward towards delighting both the business and its customers. At the end of the workshop, you are left with lots of discussion notes, post it notes, sketches, wireframes and clear design goals. Everyone says their goodbyes and returns to their respective teams. Now … after a week or so, the workshop energy and focus starts to slowly fade. It becomes harder and harder to remember the key parts of the UX Vision and it becomes challenging to communicate that out to other key stakeholders who were not at the workshop.

So the question is:

How do you document and communicate the UX vision to the organisation so everyone is on the same page on what needs to be delivered?

After teaching at Savannah College of Art and Design for close to five years, I found myself with over four hundred alumni, and I keep in touch with a large quantity of them through email. A strange pattern started to become evident in our communications: a lot of them are unhappy.

Our Passionate Youth

Students would contact me and describe how miserable they were with their jobs, asking for advice on new career paths or even entirely new professions. It wasn’t that their bosses were mean, or that their working hours were awful; it wasn’t even the larger issues we’ve all dealt with in the business context, like the misappropriation of designer as stylists, or the prioritization of technologists over designers. Instead, I began to hear how the benefits of ‘flow’ and ‘being creative’ and ‘solving really hard problems’ were being grossly outweighed by feelings of insignificance and irrelevance. My alumni were at the forefront of design, working at major consultancies and the heart of the Fortune 500 – and they didn’t feel like their work was meaningful.

I think many of us have confronted a similar feeling in their career, and we’ve rationalized meaning into our jobs. We’ve told ourselves that we were making the world a better place by making objects of beauty, or by increasing the usability of software, and that seems to satiate the concern, at least temporarily. Or, we’ve embraced management, and tried to mentor and guide other designers who were struggling with skills, theory, or career path development. And in many cases, even if these things didn’t pay off, we’ve stuck with jobs that we weren’t particularly fond of, because we had mortgages to pay and families to feed.

But for my alumni, and for the graduates that make up the 55 million millennials in the US, this doesn’t cut it. As a gross generalization, they don’t want the kids, the house, and the two car garage, and so they also don’t want the platitudes of staying the course and doing what you don’t want to do and it’s just a job. Simply, they feel entitled to a career that’s important and that contributes in a meaningful and powerful way to build a better world around them. As they find themselves in a workplace where they are designing diapers, or websites, or even the coveted jobs of designing cars and shoes, the realities of a career supporting destructive, consumptive behaviors just doesn’t seem to jive. And as they watch the banks collapse and the government flounder and the earth implode, they seem to experience a sense of personal longing – a longing for a job that matters.

This isn’t hyperbole. This is the conversation I’ve had over, and over, and over again with my alumni, and I’ve come to a simple conclusion. The creative class of 20-25 year olds won’t be satisfied playing under the old rules. Their goal and primary motivator isn’t financial capital or social capital; it’s personal recognition of meaning. This isn’t surprising, given their cultural backdrop of reality TV and Facebook profiles, and I’m certainly not the first to point this out. But the most interesting part of this desire for recognition is how it relates to the need to right the wrong and fix the broken. There’s a need – an entitlement – to work on big projects, projects with impact, and to be publically and loudly recognized for their creative efforts.

The creative class of 20-25 year olds won’t be satisfied playing under the old rules. Their goal and primary motivator isn’t financial capital or social capital; it’s personal recognition of meaning …  a need – an entitlement – to work on big projects, projects with impact, and to be publically and loudly recognized for their creative efforts.

Our Broken Educational System

Combine this pattern in the guise of modern design education, where integrated efforts between business and design are somehow seen as novel and well intentioned design educators dread the  curriculum council and petty turf war of tenure. Close to six years ago, when I was proposing a Contextual Research Methods course at Savannah College of Art and Design, the Dean of Liberal Arts essentially filibustered the course, blocking its passage through the approval process for close to a year. The reason? He felt ‘ownership’ over all aspects of research, and since no one in the design department had a PhD in Anthropology, how could they possible teach a course in contextual design research and ethnography?

These silly displays of infighting are present at nearly every educational institution in the world, and it’s against this backdrop that the aforementioned entitled students find themselves looking for direction and guidance. To be blunt, they don’t care about the credentials of their teachers; they care that their teachers are knowledgeable and passionate. They aren’t looking for incremental aspects of change that play in the context of the old guard; they see through these small steps forward in a time that requires new approaches and new passion.

There are some fantastic educational programs that have reacted to the changing space of design. New transdisciplinary efforts at Parsons have great potential; existing efforts like the KaosPilot School in Denmark serve as a template for new educational models. But these programs are the exception, and the design students graduating from schools of Art and Design are still learning the tired design-as-form-giving approaches of Bauhaus-driven foundations.

Students at universities frequently suffer the same lagging curricula, as the pace of academic change is slow. A few schools have managed to keep pace with industry, or even lead industry in a particular direction. The well known d.school program at Stanford, under the leadership of  David Kelley and Larry Leifer, and the Rotman School of Management, under the leadership of Roger Martin, have helped advance the role of designers in corporations to unprecedented levels of access, and have helped substantiate design as an independent and worthwhile endeavor.

These programs prepare students for bringing the intellectual power of design to the boardroom to solve the gnarly problems of corporate strategy.

But what if the same educational model was presented with a focus exclusively on transformation of our world around us? The students are clamoring for it, and the world is seemingly ready to embrace a model that doesn’t position corporate vs. consultancy, with the occasional NGO thrown in for good luck. This isn’t even a new idea, as it was fundamental to the design philosophies of Buckminster Fuller and Victor Papanek, and taught (to me, and countless other generations, at Carnegie Mellon) by Richard Buchanan and Craig Vogel.

Transforming the Wicked Problems

It’s in response to these students, and to these traditional problems of academia, and in the spirit of Buchanan and Vogel’s teaching that I’ve started a new educational institution: The Austin Center for Design. The program is entirely focused on Interaction Design and Social Entrepreneurship, with an explicit spotlight on designing for massive change and social innovation. The center exists to transform society through design and design education. This transformation occurs through the development of design knowledge directed towards all forms of social and humanitarian problems.

I have no misconceptions that designers can ‘solve’ massive problems, or even approach them on their own without collaboration from other disciplines. But I feel strongly that designers make great agents of change and can champion new and novel approaches to old and tired problems. The best indicator of design success, in my experience, is a passion to make an impact, and I see a generation that is wildly passionate about addressing social problems.

I feel strongly that designers make great agents of change and can champion new and novel approaches to old and tired problems … and I see a generation that is wildly passionate about addressing social problems.

I hope other educational institutions can escape from the lackadaisical pace of academic change, and I intend to publish the entire curricula that is developed at AC4D to help support other like-minded faculty who may be stuck pushing the curricula-change rock uphill. The problems to tackle are big enough to escape ego; one school can’t possible support the talent necessary to mitigate the large-scale social problems of poverty, equality of education, or health and wellness.

These are problems worth solving.

Top image: One Laptop Per Child / cc attribution 2.0

I love the TV show Myth Busters because it challenges what I think is true. In the show, popularly held myths are tested in an entertaining and somewhat scientific way. My favorite part of the show, other than the explosions of course, is when my beliefs turn out not to be true. This always keeps me open minded, and focused on reality. I also enjoy being on the other end – exposing myths as unfounded. This is the perspective I am taking for this article. Specifically, I will be highlighting five common myths related to online (unmoderated) usability testing, and explaining why I think these myths are not true.

The motivation for this article is to help UX researchers keep an open mind about online usability testing. There are some researchers who have been using this approach for years and find it useful (in certain situations). Others are new to it, and wanting to learn more about its strength and limitations. Finally, some UX researchers have already formed an opinion about online usability testing, and deemed it not useful for a variety of (unfounded) reasons. I hope by exposing these myths, we (as a UX community) can evaluate this tool based on its actual merits.

1.    There is too much noise in the data to be trustworthy

Perhaps the most common myth about online usability testing is that the data are not very reliable. Some people will say that the participants don’t try very hard, get distracted in the middle of the study, go through the study as fast as they can to get the incentive, or even misrepresent themselves to qualify for the study (in order to get the incentive). This is certainly true, and generally occurs in 5% – 10% of all cases. But, there is good news!

There are some very useful techniques to clean up the data. Identifying (and removing) “mental cheaters” is not very hard. These folks behave in highly predictable ways. “Flat lining” is when someone answers all 1’s or 5’s on a 5-point rating scale.  “Speed traps” are one way to identify these folks (see question 6 in the figure below) by checking to make sure they are actually reading each question. Consistency checks can be used by re-wording questions, and analyzing the consistency of responses. Removing unrealistic task times is also very easy to do, and can help deal with the participants who went home for the weekend in the middle of a task. Also, screening questions can be written to minimize the number of fraudulent participants (those that misrepresent themselves) who take part in the study.

Bottom line, the data from an online usability test can be just as reliable as a traditional lab test. The only difference is that a little cleaning up needs to happen before jumping into the analysis phase.

2.    You can only collect quantitative data

I am always surprised when people tell me that online usability testing is only useful for collecting basic metrics such as task success, completion times, or satisfaction. For some reason, people assume that just because you have a large sample size, you can only collect quantitative data. Online usability studies just don’t make sense if you want to collect qualitative data about the user experience. This cannot be farther from the truth.

One of the beauties of online usability testing is that the researchers can collect a rich set of both quantitative and qualitative data about the overall user experience. For example, if a user has difficulty with a specific task, they can provide comments as to why they had difficulty with that task, or what they were expecting to happen at different points along a transaction. Qualitative data, usually in the form of verbatim comments, can be collected at any point in their experience. The user can even be prompted to provide verbatim comments when they exhibit certain behavior, such as abandoning a transaction or deviating away a desired navigational behavior.

Verbatim comments are not only easy to collect, but they are becoming much easier to analyze. There are many tools available now to pick out patterns in verbatim responses. Word clouds (see below) are one example to way to get a quick sense of the key patterns in verbatim responses.

3.    Online usability testing does not work well during the design phase

Most people think of running an online usability study only after the product has been built. It only makes sense to run an online usability study as part of a summative evaluation, and doesn’t really fit in during the actual design phase. Online usability testing can’t inform the design, takes too much time, or is too expensive to conduct during the design phase. These are all untrue.

Online usability studies can be set up within a few hours, and data collected in a matter of hours. We have set up, launched, and analyzed data from an online study within the same day. Not only can a study be set up quickly, but it can help answer questions that typically come up in the design phase. For example, if there is a question about preferences around navigation method, labeling, or look and feel. These and many other design preference questions cannot be reliably answered with a small sample size. Sometimes, we need to gather data quickly from hundreds or thousands of users in order to validate significant design decisions.

4.    It only works with websites

One of the things that researchers quickly point out is that online usability testing can only be used for evaluating websites. It simply doesn’t work for software, voice response systems, mobile, consumer electronics, and toaster ovens. While a vast majority of online usability testing does focus on websites, it doesn’t have to.

Over the years, we have used an online usability testing approach to evaluate non-web interfaces. Conceptually, it is still the same method. The participant is asked some questions, given some tasks, and provides feedback about their actual experience. The only real difference is that the participant’s behavior is not being tracked. But, a lot of useful data can be collected about their experience, such as whether they were successful in completing their tasks, how long it took them, what they felt about their experience, and where they had difficulty. So, even though you might be giving up a little behavioral data, there is still a lot to be learned about their experience.

5.    Online usability testing costs too much

Some people say that they would love to do more online usability testing but it is simply too expensive to use on a regular basis. It certainly can be pricy, but fortunately, in the last few years there are a variety of tools that allow you to run online usability studies for not a lot of money. There are various self-service providers to allow you to set up and run your own online usability study. You only need to pay to access their technology. Visit www.measuringux.com to see a complete listing of vendors.

If budgets are really tight, there is a way to run your own “discounted” flavor of an online usability testing for free, or practically free. By taking advantage of some online survey tools (such as Survey Gizmo or SurveyMonkey), and a little html and java script, you can literally create your own online usability study for close to free (see the figure below as an example of the “homegrown” approach). While you will give up a little data and functionality, it can be useful in those situations when you have no budget to run an online usability study. Visit www.measuringux.com to see more details about how to create a discounted online usability study.

Try it for yourself

I may or may not have convinced you that these myths are untrue. Regardless, I would encourage you to consider online usability testing as part of your user experience design and research efforts. In doing so, you will discover its’ strengths and limitations. After all, every user research method has its own strengths and limitations. Online usability testing is no exception.

Editorial note: Interested in learning more about this subject? Bill recently co-authored Beyond the Usability Lab: Conducting Large-Scale Online User Experience Studies.

Top image: Brad Montgomery / cc

In 1973 the renowned author and member of the so-called ‘Big Three’ of science fiction Arthur C. Clarke decided to put his opinions of successful predictive storytelling into law. Behold his third and most famous law: “Any sufficiently advanced technology is indistinguishable from magic.” Now, I’m going to go out on a limb here and modify Clarke slightly to read “Any sufficiently designed interaction is indistinguishable from magic.”

So, what do we think?

In Clarke’s books, 2001 A Space Oddyssey for instance, his writing prowess shone when he was describing deep future technologies. It was critical to him and his contemporaries to make sure that the far out tech was somehow based on the most advanced existing tech of the time, only extrapolated a few decades or centuries. Thus “predictive storytelling”; essentially, taking what we have now, and imagining what it will be in the future, based on extensive knowledge of current research. The better the storyteller, the more magical the future seems.

“If design can be a way of creating material objects that help tell a story, what kind of stories would it tell and in what style or genre? Might it be a kind of half-way between fact and fiction? Telling stories that appear real and legible, yet that are also speculating and extrapolating, or offering some sort of reflection on how things are, and how they might become something else?”
- Julian Bleeker of The Near Future Laboratory

Pretend for a second that you can predict the future

In fact, you have just been sent through a mental wormhole 100 years into the future. What do you see? How are people communicating? Traveling? Eating? Now bottle those visions up and bring them back to our present-day with you. Oh, what’s that, you can’t? Why not? The technologies don’t exist, you say? Hmm… Ok, how bout this – come on back and write some stories about it. Or better yet make us a ton of prototypes that each hint at some part of the future! How do you suppose an interaction designer might take on this challenge differently from say Arthur C. Clarke? This is what interests me: predictive art and design that is essentially indistinguishable from magic. I’d like to clarify my meaning of the word “magic” a bit. Historically magic has meant many things, ranging from illusionists’ glamorous stage performances to wizards and witches casting spells. The “magic” I’m referring to is more of an abstract concept I suppose – one that creates a feeling of wonderment in its audience by exhibiting some seemingly impossible or supernatural feats.

The multimedia duo SWEATSHOPPE recently delivered some magic to the streets of New York. Consisting of Bruno Levy and Blake Shaw, SWEATSHOPPE “works at the intersection of art, music, and technology” all three of which are resonant in this digital-light-painting evidence video.

More magical interactions

If we push interactions to be more and more magical, they will begin to be indistinguishable from the future and from magic itself. I have been referring to existing examples of this effort as “Fringe Design”. Fringe, as in living on the outskirts of “the ordinary”; pushing the boundaries of the imagination, the visual vernacular, and the plausible, without batting an eyelash. Fringe Design should be embraced for its foretelling abilities – not every crazy technology ridden invention predicts how we will live in the future, but it MIGHT! Julian Bleecker, co-founder of The Near Future Laboratory puts it best in his essay, Design Fiction: “Science fiction can be understood as a kind of writing that, in its stories, creates prototypes of other worlds, other experiences, other contexts for life based on the creative insights of the author. Designed objects — or designed fictions — can be understood similarly. They are assemblages of various sorts, part story, part material, part idea-articulating prop, part functional software.” Further on he states that “design fiction objects are totems through which a larger story can be told, or imagined or expressed. They are like artifacts from someplace else, telling stories about other worlds.”

Neurosonics Live from Chris Cairns recalls the decades-old dream of realistic, interactive 3D holograms.

They say good design is invisible

What we’re talking about here is the exact opposite (although it may occasionally involve invisibility cloaks or some such related thing). We’re talking about design as spectacle. Art as predictive storytelling. The practicality of this type of work may not seem immediately apparent, but I urge you to think deeper into the future, when the things that these works hinted at decades or centuries ago, are an everyday reality.

Good design is about the voyage

As always, with good design, it’s about the voyage; the process by which the idea evolved. As J.J. Abrams puts it in his article for the April 09 issue of Wired Magazine, “the buildup to a magic trick’s big flourish—is as much of a thrill as the result. There’s discovery to be made and wonder to be had on the journey that not only enrich the ending but in many ways define it.”

Science fiction is largely based on the notion of an alternative future world. Consider this futuristic present the big flourish of the magic trick and everything up until then, the buildup. Since time isn’t restricted to a fixed line in the sense that there is only now and the future, but infinite points in time in between, this journey is happening now and will continue to happen forever. It’s up to designers to consider our role in delivering magical flourishes for the alternate worlds of tomorrow. Without props, there could be no magic trick. That is certainly not to say, however, that by simply having props, you have a magic trick. The props serve as vital, tangible ingredients in the total experience of a trick, but they alone do not add up to a series of interactions that comprise an experience. Claiming such a thing would be to ignore other crucial components such as the magician, the stage and architectural context, the crowd of fellow wonderers around you, and perhaps above all, the theatrical trickery of a good magic show. This is not to say that well considered interaction design MUST employ trickery, but what if it did share more similarities with magic shows? How could we begin to think about designed objects as part of a “service and people” economy the way props are part of a magic performance? Consider the three acts of a classic magic trick (as portrayed in the movie, The Prestige):

• Act 1: The Pledge -  The magician shows you something ordinary, but of course, it probably isn’t.
• Act 2: The Turn – The magician makes his ordinary something, do something extraordinary. Now you’re looking for the secret, but you won’t find it.
• Act 3: The Prestige – This is the part with the twists and turns, where lives hang in the balance, in which you see something shocking which you’ve never seen before.

What if designed experiences followed this pattern (minus the life-threatening bits in the third act, of course)? Perhaps designers can integrate aspects of this time tested process into the design of every things. To take users to a new depth of wonderment where anything seems possible.

Arthur C. Clarke knew that science fiction fans were seeking “immersive fantasies. They wanted warmly supportive subcultures in which they could safely abandon their cruelly limiting real-life roles, and play semi-permanent dress-up…”

as Bruce Sterling notes in his essay, Design Fiction. He knew that in order to make his fantasies truly immersive, though, he would have to stay somewhat grounded in contemporary notions of the future, reflecting upon today while extrapolating into tomorrow. To inspire readers to speculate on how things are and wonder how they might become something else. Maybe, even to motivate the designers out there to take charge of defining the steps in between today and tomorrow.

Microsoft plans to have its Project Natal available for Christmas later this year, proving that experience designers ARE working on delivering tomorrow to today’s holiday shoppers. If it works half as well as all the press videos make it look, it will be the most magical gaming system of all time.

Interacting with a thing rarely happens all at once and then is over

It tends to happen over time, or at least can be broken down into a series of interaction events. Each of these events presents an opportunity to elicit the same sense of wonder and joy of a good magic trick, or the end of a great chapter in a science fiction novel. “What will happen next?” the makers of these things want you to ask. “How will this end?” “How does it all work?” Magic and science fiction both typically do a wonderful job at this, but what about design? Why can’t our everyday experiences push our mental boundaries of what is possible today and make us wonder more often – what will tomorrow be like? What will the next century be like? I propose we follow Julian’s lead and “throw out the disciplinary constraints one assumes under the regime of fact” and allow our minds to wander. Wander to a place where robots, ray-guns, time machines, artificial intelligences, nanotechnology, and magic abound. Let us all wander, to tomorrow.

Top image: The Prestige

A few months ago we set up five Flickr groups around several UX topics. Every month we will try and make some sense of the uploaded material. This month we selected the UX Errors group and will look at examples of issues that arise when proper attention isn’t paid to two very important components of successful user interface design: Perceived Affordances and Designing for Task Flow.

Perceived Affordances

In The Design of Everyday Things Don Norman introduced many designers to the concept of affordances, which he would later clarify as “perceived affordances”, or the actions the user perceives as being possible based on how an object is presented. For example, if your design includes a “button” make that button look pushable.

Posted by Adam Connor

Push me. Wait… you can’t

The ATM example suffers from some misrepresentation. The labels for the physical buttons are styled with bevels, commonly used to represent buttons in GUIs. As a result, many users try to press them, only to realize after a few attempts that the buttons are actually to the right and left of each label. In this case, objects give off an inaccurate perceived affordance.

It’s not uncommon for ATMs to have physical buttons and digital label displays. In this situation, it looks like someone thought that adding a little visual “excitement” to the labels would be an improvement. If simple, plain text labels had been used, people would most likely have experienced less confusion.

Posted by Olivier Lorrain

Where’s the Button?

In the gas pump example people tended to press the octane number, as evidenced by the fingerprints around the “87″, when choosing a fuel. This happened despite the “Push to start” label on the actual button and the arrows pointing to them. In this case, there isn’t enough emphasis given to the real button in comparison to the large yellow square label. Which begs the question, why not just make the yellow square the button to begin with?

Posted by Andreas Popp

Which way is up?

In our last perceived affordances example, we have three components, all with a triangle/arrow pointing up, all beveled from the panel on which they are presented. Which one do you push to direct the elevator up?

The two arrows on the top are lights, which indicate the direction the elevator is traveling, while the lower button is the actual call button. Many users pressed the lights in order to call the elevator. Why? Most likely because the bevel caused people to perceive them as buttons. Also, the lights have a higher visual significance than the smaller triangle on the actual button. Had there simply been no bevel around the two lights, it’s likely there would be less confusion and more people would find the real button first. That’s not to say that the button itself couldn’t use a bit more visual prominence in it’s display too.

Designing for Task Flow

When I was in grade school there was an exercise we did where each student had to write instructions on how to construct a peanut butter and jelly sandwich. The teacher then attempted to construct the sandwich according to those instructions verbatim. If an instruction left something out, small details like removing a slice of bread from the bag containing the loaf, the teacher would stand there and act stumped until the student modified their instructions to be more specific.

The purpose of the exercise was to give students an appreciation of the fact that even simple tasks are comprised of a multitude of steps, and that finding the right level of detail and sequencing for those steps is critical to user’s success and satisfaction.

Posted by Eugenia Ortiz

Follow the Arrows

In the example to the right, there appears to be some attempt at sequencing in the digital display as well as from the large “2″ at the bottom of the device. In this case however, some considerate and/or frustrated individual(s) has taken it upon themselves to try to make up for the devices inadequacies by adding additional instructions and labels.

Posted by Frank Farm

Easy as 1, 2, 3

Our second example also uses numbers, in addition to a list of instructions, to provide guidance. Yet the arrangement of components and labels gives a perception of chaos that even a well written set of instructions can’t overcome.

Both of these interfaces exemplify that in user interface design, identifying a sequence of actions is not enough. Numbers and lists, while somewhat helpful, should be used in conjunction with a logical and sequential arrangement of controls and inputs.

Johnny Holland’s Flickr Groups

Observations are a critical tool in any designer’s tool set. They provide us with in-site on things we should do, shouldn’t do and could do better. Many of us photograph our observations to preserve them, to keep them as reminders, learning tools that won’t be lost in the background when our next big breakthrough comes along. We also share our observations so that we can learn from and educate others. So that we can build a stronger dialog and further conversation.

To that extent, Johnny Holland has established five Flickr groups, so that we can share, discuss and learn from eachother’s observations.

In addition to the UX Errors group, where we collect examples of design decisions that have a negative impact on individual’s interactions with a product and overall user experience, and from which this month’s examples were taken, we have:

• Daily UX: collecting pictures of everyday user experiences, good or bad.
• UX Sketches: collecting sketches of products, interfaces and ideas
• UX Patterns: collecting examples of interface and interaction patterns.
• UX Events: collecting photos from UX conferences and meet-ups around the world.

Like any collaborative effort, these groups are what we make of them. So please join, share your photos, comment and discuss. And if you have any ideas on how we can improve the groups or better utilize them, please let us know.