With their February 24^th revelation of more features in the upcoming OS X Lion operating system, Apple may have taken its first steps toward an unfamiliar future… a future in which the file system does not exist.

Credit for this observation goes to Mike Rundle, who tweeted about being able to imagine “a future in which the Finder does not exist”. Documents would be associated with the apps that created them, like on the iPad. Mike went on to describe his vision in more detail, a vision in which users simply have apps. “Documents associated with them appear magically. Presto.” While this might sound like some kind of user experience utopia, I have a grave concern that eliminating a file system in this manner misses a huge audience.

Us.

While opening Pages to work on the family newsletter might make sense for casual home users of a computer system, it does not make sense in a professional context. In the professional world, we work on projects. Projects are composed of many different types of files. And yes, we might have the same apps open all day, but do we want to be forced to duplicate a hierarchy of information in every single application?

No.

Besides, “projects” are just one type of organizational scheme. As a user experience designer, I’ve seen a lot of professionals in other fields organizing a lot of stuff in a lot of different ways. So even attempts at inter-app organization around the concept of a project, such as Microsoft’s Project Center, are not effective replacements for an infinitely flexible organization scheme like simple folders.

Some Wheels Need Reinventing

The conversation that Mike’s comments sparked led us both to the conclusion that we still need a high-level organization system of some kind. And that is the challenge. It’s a challenge because that problem has already been solved by the file system. The challenge is to solve it better.

At Interaction11, Tim Wood called for designers to reject the “complacency artifacts” of the past, design patterns that have lost relevance in the modern world but continue to be used simply because that’s how things are done. He encouraged us to be bold enough to reinvent wheels that need reinventing, and that’s exactly where we’re at with file systems.

Gestural user interfaces, effortless portability, and ubiquitous network access… All these things and more are redefining how people interact with technology. UX designers need to recognize this and push themselves beyond the limits of our vision. Yes, we absolutely must continue observing people interacting with technology, analyzing those interactions, and synthesizing solutions that work in context. But what’s even more important now is that we rely on our raw creativity for that last part. There are old problems out there that need to be solved in new ways, and the file system is one of them.

——
Header image courtesy of Tim Wood.

The conclusion of the Nielsen Norman Group’s April 2010 study of iPad usability is that it has problems and more standards are the solution. Yes, the iPad is imperfect, but resorting to standards as the solution is an antiquated reaction that fails to consider how interactive systems have evolved. We’re not Usability Engineers anymore (not most of us, anyway); we’re User Experience Designers. Experience is more than just usability.

I’ve covered this ground on Johnny Holland before. Just after I got my iPhone I came to many of the same conclusions Nielsen did about the how the iPhone is difficult to learn. But here’s the thing; I didn’t stop there. I talked about how some of the factors that made the iPhone difficult to use also made it fun to use, which is why it has flown off shelves since it was introduced. As I got used to it I began to think more about how playfulness was more delightful than pure usability in some contexts and vice versa. Something I use occasionally for very specific tasks delights me if it is simple and usable. But something I use often or for more amorphous tasks that is simply usable will either provoke no emotional response or, at worst, will become tedious. In that context, a more playful interaction style will keep me engaged and might even lift my mood a little.

This is the perspective from which I’ll look at what Nielsen found, identify where it’s valuable, and point out where it’s a little myopic.

“Wacky Interfaces”

Wacky. Yes, “wacky.” As in, “Isn’t it cute how kids these days are trying to create beautiful experiences.” Beauty does not require an unusable interface, but a beautiful experience might ask you to engage with it a little more deeply through a lack of obvious affordances.

For more than a decade, when we ask users for their first impression of (desktop) websites, the most frequently-used word has been “busy.” In contrast, the first impression of many iPad apps is “beautiful.” The change to a more soothing user experience is certainly welcome, especially for a device that may turn out to be more of a leisure computer than a business computer. Still, beauty shouldn’t come at the cost of being able to actually use the apps to derive real benefits from their features and content.

He almost gets it. No, the iPad is no business computer, and that’s exactly why beauty is an asset. People will, much of the time, interact with this device in order to have an experience rather than complete a task. Nielsen’s wholesale discounting of beauty fails to take into account that some apps will be experiential and content based while some will be functional and task based. Engaging with a system is not what people want to do when they have a task to complete. That’s when basic usability is more delightful.

Long-standing GUI design guidelines for desktop user designs dictate that buttons look raised (and thus pressable) and that scrollbars and other interactive elements are visually distinct from the content.

The iPad does not have a Graphical User Interface but a gestural one. GUI design guidelines do not necessarily apply when users can interact directly with the content.

For the last 15 years of Web usability research, the main problems have been that users don’t know where to go or which option to choose — not that they don’t even know which options exist. With iPad UIs, we’re back to this square one.

The iPad is also not the Web. Interacting with apps is completely different from interacting with websites. Most apps have far fewer options than the average website, lessening the potential for confusion. On top of that, people use apps in a much more focused way than they use a website. Users can access the entire Web when they open their browser, but when they open an app they choose to focus on that app’s content and functionality only. In that context, a more deeply engaging, exploratory design can enhance the user’s experience.

“Inconsistent Interaction Design”

I take issue with this finding because Nielsen evaluated multiple applications. That’s like saying it’s bad that Microsoft Word and Adobe Photoshop are inconsistent. They allow completely different audiences to accomplish completely different tasks. He considers it confusing that the same gesture affects the same type of content differently in different apps. When there’s a limited gestural vocabulary (and there has to be) and a diversity of contexts, it’s easy and usually risk-free to experiment with figuring out the correct gesture if you get it wrong the first time. And because it’s gestural, it’s inherently playful and fun. It’s not a chore like trying to parse Word’s menus or toolbars.

Nielsen says that iPad UIs suffer from the “triple threat” of low discoverability (non-obvious controls), low memorability (difficult to remember inconsistently applied gestures), and accidental activation. I agree with the first and the last, mostly. Non-obvious controls can encourage exploration and playfulness in some contexts, but they can be frustrating in others. Accidental activation is certainly annoying, but it’s usually easy to deactivate whatever was activated. That problem in particular I think is due to the absolute newness of the apps and the platform. At least two iPad developers I’ve heard from indicated that they changed the design of their apps once the iPad was released.

The second problem he identifies, low memorability, I completely disagree with. My pre-literate two-year-old daughter knows how to unlock my iPhone & iPad, navigate to her favorite drawing app, launch it, draw with it, and change the various options. It took very few demonstrations before she learned this. If you look on YouTube there are videos of small children expertly navigating iPhones and iPads. You show them how to do it once, they do it, and they remember it.

The link between physical motion and cognitive development (especially in children) is well established in cognitive research, making gestural UIs much more easy to remember than your typical desktop GUI. On top of that, the number of gestures that are possible is pretty limited. Even if you don’t perform the correct gesture first, it won’t take long to figure out what the right one is.

“Crushing Print Metaphor”

Nielsen again complains that iPad apps are not like the Web.

The current design strategy of iPad apps definitely aims to create more immersive experiences, in the hope of inspiring deeper attachments to individual information sources. This cuts against the lesson of the Web, where diversity is strength and no site can hope to capture users’ sole attention.

My friend Pete Barry likes to talk about the value of experience. The reason people choose to consume content through these “limited” apps is because the experience they provide is valuable to them in some way. That experience is a benefit rather than a drawback. Besides, the open Web is just two taps away.

“Card Sharks vs. Holy Scrollers”

Nielsen references Jef Raskin’s differentiation between “two fundamentally different hypertext models,” Cards and Scrolls, indicating that iPad apps mostly fall into the Card model. On a Card, all the interaction occurs on a fixed size canvas that is swapped out to provide access to more content or functionality. And I’m sure I don’t have to tell you what a Scroll is.

Nielsen said:

There’s no real reason we can’t have both design models: cards on the iPad and scrolls on the desktop (and phones somewhere in the middle). But it’s also possible that we’ll see more convergence and that the Web’s interaction style will prove so powerful that users will demand it on the iPad as well.

If I read that right, I actually agree with him. The iPad doesn’t have to force all apps to subscribe to one model; each app can use whichever model is most appropriate for its context of use. I’ve even seen some apps that mix the models, like Early Edition. This newsreader arranges RSS feed articles like a newspaper, with a home page and different pages for each individual feed. Wherever an article appears on any of these pages, you can actually scroll in place to get a sense of what it’s about! Granted, this is something users are likely to discover accidentally, but it’s a pleasing, delightful interaction nonetheless.

Nielsen’s Recommendations

This is what really gets me going. And not in a good way. He has four, but they really roll up into three:

• Make iPad UIs look more like GUIs
• Make iPad interaction design more like the Web
• “Abandon the hope of value-add through weirdness.”

And yes, that third is a direct quote. In 2010. Beauty isn’t weird. Compelling interactions aren’t weird. Both of these are critical components of modern interaction design, where designers seek to go beyond simple usability and create positive emotional experiences that build loyalty and emotional attachment. What is perhaps most confusing about these recommendations, though, are the first two. Jakob Nielsen is a smart guy, and clearly the iPad exists within entirely different contexts of use than a desktop GUI or a website.

What I really want to know is this: why does Nielsen feel that iPad apps should be designed for contexts they won’t be used in?

—–

Header image by nDevil /CC 2.0

I live and breathe user experience design, and yet it took me two years to get myself the device referenced by almost every single presentation about user experience since 2007… Apple’s iPhone. My reasons were very specific and perhaps boring, but what is interesting is the perspective this wait has afforded me. Since it was released, the iPhone has grabbed an astonishing share of mobile Web traffic, been regarded as a “game-changer” in both the design and business worlds, and has even been referred to as the “Jesus Phone.” Now that I’ve owned one for two weeks I’ve developed a different perspective. The iPhone is surprisingly difficult to use, but it sure is fun! And that is why it’s a game-changer.

A Lack of Affordances Leads to Low Learnability

Learnability contributes greatly to the usability of a system. If a system is designed for a specific context, it should be easy for people in that context to approach it, assess its controls, and manipulate it. Granted, learnability isn’t everything, but when it’s tough to figure out how to do things you’re on the express bus to a frustrating experience. There are two things about the iPhone that contribute to its difficult learnability. It lacks physical affordances and suffers from inconsistent visual cues.

Gestural user interfaces (UIs) are the 21st century’s version of the command line interface… they’re really fast and easy provided you’ve memorized a bunch of commands. This is fine for those who are accustomed or inclined to explore a device, but many people just want to check their calendar, write an email, or make a grocery list. These people will react to what they see on the screen rather than explore possibilities, which leaves them out of luck with a gestural UI.

The iPhone’s featureless touchscreen is Don Norman’s proverbial glass door. Apple has done a stunning job of making things that are pressable look like they’re pressable, but that will never be as effective as an actual button. With physical, simple buttons we can rely on motor memory to manipulate a device without paying attention. But the iPhone’s buttons are highly contextual, which forces us to pay attention to the device to remain aware of its context even after extensive use of the system. The “problem” is that the iPhone is a convergent device, a device with multiple functions. With 50,000 apps, you might even say infinite functions. The only way to build a device that serves 50,000 different purposes is to make it almost entirely free of physical affordances. Of course, the big value proposition of the iPhone is that it is the first mobile device to achieve an effective convergence.

Pressing a button is an action that a gestural UI can communicate visually, but there are a number of other actions that have no visual cue. Direct manipulation gestures such as tap (on something other than a button), double-tap, tap-and-hold, swipe, and pinch/zoom are far more difficult to communicate. These rely on user experimentation and memory.

Even worse are the modal gestures such as shake to undo and swipe to delete. If users discover them at all it’s usually by accident. They don’t map to anything (outside of an Etch-a-Sketch) and there are no clues to indicate that they’re available. Being mentioned in a WWDC keynote does not count as a clue.

My phone displayed this message several times while I was simply using it. The message is without context, and what’s worse reveals an important feature without showing how to access it on purpose.

The red Delete button also showed up unexpectedly. I had no idea how I made it appear. Its appearance made me feel uneasy because I didn’t want to delete anything.

Inconsistent Visual Cues Don’t Help Either

Apple has gone to great lengths to make the UI consistent, even publishing the iPhone Human Interface Guidelines, but some inconsistencies remain. Application buttons can have labels or not. Some applications, like TweetDeck, AP Mobile, and others, obligingly label their buttons:

An example of an iPhone application with button labels.

Others, mostly Apple applications, do not:

The iPhone Mail app

Does the circular arrow mean reload like in Safari? Or reply? If it means reply, what does the other arrow mean? Labeled buttons communicate their functionality much more clearly. (The circular arrow does mean reload, but makes no sense in the context of a message. The swoosh arrow does mean reply.)

The landscape keyboard, despite being a basic device function, isn’t supported by all applications. When it is supported, there are no visual or other cues that indicate it. Not only is it difficult to learn when the landscape keyboard is available, cues as to its availability are stored in only one place, user memory.

Even the iPhone’s implementation of its standard gestural interactions is inconsistent. This is most frustrating on simple interactions like tap. There are obvious tap targets like buttons and non-obvious targets like received calls, tweets, emails, etc. In some cases, a tap on a non-obvious target means “open” or “get detailed info.” But in others it means “take action.” The worst example of this is the Recent Calls list. I can’t tell you how many times I’ve accidentally called someone when what I wanted to do was get more details about the call. Yes there is an arrow button, but it’s on the right side away from my focus. Other applications (like Mail) have trained me to tap an object to get a detailed view of it, so my natural tendency is to tap the contact name or number.

Clicking a contact name makes a call instead of revealing details about the call.

When applications do not implement buttons, device functions, and non-obvious gestural interactions consistently, this increases the learnability problem. Not only do users have to learn and memorize what the device does, they have to learn how each application makes use of those functions! This is much less of an issue in point-and-click interfaces, which require fewer physical interactions and present most options on the screen for users to react to.

If the iPhone is so difficult to use, why is it still regarded as a game changer by both the design and business worlds? Because it does several important things right, but most of all because it’s fun.

Fun is the New Usable

As a user experience designer, I thought my job was to make things not suck. Until recently. As technology has evolved, human behavior has evolved along with it. Since behavior is the basis of user experience design, my job has evolved as well. Now, my job is to make things people love. At the 2009 IA Summit, Karl Fast articulated the value proposition of user experience design with sparkling clarity. “Engineers make things,” he said, “we make people love them.” And then he held up an iPhone as an example.

This is a crucial change, the importance of which cannot be overstated.

Play

Any new system or gadget has a learning curve, but where the iPhone differs is that the nature of traversing that curve is more fun than frustrating. You swipe and pinch and tap and shake your way to familiarity instead of pressing awkward buttons and navigating byzantine menu structures. You learn the iPhone by playing with it, which encourages interaction because humans are built to play. Even in a system like this, we could quickly be dissuaded from doing so if wrong actions had negative consequences, such as getting online or sending messages accidentally. The iPhone is mostly devoid of these sorts of consequences. The only time I’ve run into this is repeatedly calling people I didn’t want to call while viewing my Recent Calls list.

The iPhone goes further than encouraging play; it rewards play. If you explore the phone’s applications, you will often find them anticipating your needs. When viewing a video you’ve shot and press the action button, you can email it or upload it to YouTube. If you try to email it and the video is too large, it will ask if you want to send a smaller clip from the video instead of preventing you from sending it. The iPhone then presents you with the UI to trim a clip and continue with your message. The original video remains untouched. Simple, sensible, satisfying.

Effective & Delightful Convergence

On the day I got my phone, someone sent me an email that contained a physical address. The phone turned it into a link. I clicked it, got a map, and the phone asked me if I wanted directions. From my current location. I giggled excitedly.

The delight induced by how well the iPhone’s applications interact with each other is another reason for its success. This is the point at which usability and playfulness intersect. The experience of having needs not just met but anticipated creates the joy that encourages users to continue exploring. This intelligent interaction between applications is absolutely key to making a convergent device delightfully convergent.

But you can’t have a delightfully convergent device that isn’t effectively convergent. What converges are contexts of use. The interactions between applications that I described above represent relatively minor, detailed contextual shifts. These small shifts result in delight only if the device handles major shifts effectively as well.

Before I had an iPhone, I would switch major contexts by switching devices. If I got a call while listening to my iPod I’d stop it, put it down, and pick up my phone. This was intuitive to the point of being instinctual. But now my iPhone must handle that switch of context for me. If it failed to do that in a sensible way, I would think the iPhone sucks. An effectively convergent device is one that, like the iPhone, can handle major shifts in context in a way that supports the user’s transition between those contexts.

Implications for User Experience Designers

iPhones fly off the shelves despite being difficult to learn.

Why?

Because they let you do what it says you can do and they make you happy while you do it. This proves that my job as a user experience designer has evolved rather than simply changed. While it’s still my responsibility to prevent things from sucking, now it’s also my responsibility to add a little playfulness. As Kim Goodwin said in her Interaction09 keynote, we have a limited window in which to prove how valuable design can be to business. There are three ways in which user experience designers can learn to incorporate play into the systems they design.

Experience and Research Play

You can’t build playfulness into your designs without experiencing playfulness yourself. Play games and pay attention to what makes them fun. For example, the only rule in the card game Fluxx is that the rules constantly change. Completing a level in Peggle gives you the “Ode to Joy,” rainbows, unicorns, and fireworks! Use these elements as inspiration for working playfulness into your designs. You might not be able to play the “Ode to Joy” when people complete a purchase, but can you delight them in another way?

Play is a behavior. As a user experience designer, you should explore research about play and playfulness just as you’d explore research about gestalt perception or information seeking. The National Institute for Play is a good place to start. The ACM digital library has some resources on playfulness and computers. Questia has resources on play in general. (Both ACM and Questia are paid services.)

Become Familiar With Game Design

Game designers put a lot of thought into how to design a fun experience. We can learn a lot from the principles they use to make this happen. Much of game design seems to revolve around creating, sustaining, and developing a narrative. This aspect is less important to user experience designers than game mechanics and the design of casual games.

A game mechanic is anything that guides the play of a game. Most mechanics take the form of either rules or possible actions. In cribbage, players must discard two cards to the crib (rule) and they keep track of their progress by placing pegs on a board (possible action). Game mechanics translate into the user experience design world as interaction patterns. Understanding how game designers make games fun by designing pleasing game mechanics will help you design pleasing interactions. The Critical Gaming Network’s Game Design 101 has a great discussion of game mechanics.

Casual games are those that are meant to be picked up and played simply for the joy of playing them. They are always enjoyable, often compelling, but not engrossing. Peggle and Paper Toss are canonical examples of casual games. Casual game design is important to user experience designers because they place special emphasis on learnability and delightful interactions. When we design systems that are fun, delightfulness should be a side effect of interacting with them even though it is not the goal. People still have tasks to complete and we can’t let fun get in the way of that. For more on casual games, read the Casual Game Design blog as well as Nicole Lazarro’s “Why We Play Games.”

Re-Learn the Art of the Tutorial

My experience with the iPhone has led me to think that maybe fun doesn’t need to be intuitive. Maybe fun is so valuable that people will make the effort to learn a system built on fun interaction patterns. If I had the opportunity to change one thing about the iPhone, I would add a tutorial. By tutorial I don’t mean a boring list of stuff you can do with it. I’m specifically thinking of some sort of mini-game. It would introduce users to all the different gestures they can do and the contexts in which they’re appropriate, challenging them to choose and perform the right one.

Tutorials in the casual games I’ve played take one of two forms. The first is much like the mini-game I described above. This type of tutorial is composed of levels in which the goal is to learn, explore, and practice one or more game mechanics. The player then begins the “real” game. The iPhone games Isotope and TaxiBall contain good examples of this type of tutorial.

The second is the in-game tutorial. In games with this type of tutorial you simply start playing. Early on the game will put you in simple situations that require you to use one or more of the game’s mechanics. The game will then display a short description or demonstration of the mechanic you need to use to get over the current hurdle. The frequency with which the game shows these descriptions decreases over time. The iPhone games Spore Origins and Rolando contain good examples of this type of tutorial.

Both types of tutorials have their advantages and drawbacks. Mini-game tutorials are very focused. They allow users to learn everything at once. They keep out of the player’s way as they play the game. But what mini-game tutorials lack is context. In a game, context is less important because the world is rigidly defined. But in real-world systems, context is key to good user experience design. In-game tutorials are all about context, but they interrupt the flow of play. This is less of an issue in a game than it is in real-world systems. In a game, the frequency and temporal location of tutorial elements can be highly controlled. They appear when players expect them to appear, when beginning a game. The contextuality required to make these work in the real world means that they could interrupt important tasks and cause frustration.

Casual Game Design has several good articles on game tutorials if you are interested.

The Way Forward

I strongly believe that play is an integral part of the future of user experience design, and I am looking forward to making that future happen. To do that, I’m going to take the words of Mary Poppins to heart:

For every job that must be done, there is an element of fun. You find the fun and SNAP! The job’s a game!

I dub this the Mary Poppins Principle, and I challenge you to use it to find the fun in the jobs that your users must do. But for now, go have an ice cream cone. You deserve a treat for reading this whole thing.