Agile has a relatively short history in the broader view of software development. Integration of User Experience into Agile has an even shorter history with relatively few stories of overwhelming success. Over the last eighteen months, we at TheLadders have had some successes—and some failures—in our foray into a post-waterfall way of developing elegant, efficient and sophisticated consumer-facing software. This is our story.

Who we are. What we do.

TheLadders is an online subscription job service that helps professionals, recruiters, and employers connect to fill positions that earn $100,000 or more. Our execution team consists of product managers, developers and user experience practitioners. The UX folks fall into three disciplines: interaction design, visual design, and copywriting. The UX team’s work spans both pre-paywall acquisition and conversion marketing (e.g., banner ads, landing pages, email campaigns, etc.) as well as post-paywall product design (e.g., job search, applying to jobs, finding candidates, etc.).

Prior to undertaking the transition to Agile, the UX team was a shared service—in essence a pool of resources assigned to projects based on bandwidth and capacity to whichever business line had the next need. We worked in a traditional waterfall style with three- to nine-month release cycles, big upfront design, late-stage user testing and validation. We produced very thick functional and design specifications destined for explicit handoffs to a development and quality assurance team. We were happy with this, in that we didn’t know any better. This is the way we’d always worked, and with little to no knowledge of the Agile movement we were meeting our commitments to the business.

And then, one day, it happened.

In a unilateral move, the development organization announced that we were switching to an Agile software development approach. There was no discussion with the other disciplines and no guidance about how the UX team would integrate into this new way of building products. All of this was beside the point since Agile was sold into the organization with promises of better products, faster releases to market, tighter validation with our customers and the ability to pivot the organization on a dime. Who could argue with that?

The expectation of the UX team was that we’d “figure it out” as we went along but the myths, stories and case studies available for learning were not very promising. Initial, surface-level research into the topic revealed that while many people had implemented solid starting points, there didn’t seem to be a single organization that felt it had truly found a way to marry the iterative Agile process with the needs and outputs of a user experience practice.

The deeper we dug into the practices espoused in the Agile Manifesto, the clearer it became that our comfortable “design phase” was going to change dramatically. That realization triggered a flood of new questions:

• How do we set expectations about the work we will now generate?
• Will it be of high enough quality for the business and for us?
• As you slice projects down into iterations, how do you maintain focus on the bigger vision?
• How do you keep the development teams busy each iteration?
• Will the business accept lighter product iterations, and, if they don’t, who will be blamed?

Getting out in front of the change.

Google was the first tool we reached for. Search after search led to the same handful of resources (Jeff Patton, Desiree Sy, Salesforce and Citrix case studies) who all preached a similar approach of staggered sprints and visual display of the stories being developed (storymapping). Google also returned many stories of failure and, worse, vitriol towards designing in an Agile environment. Awesome.

Not deterred, we expanded our research to interviewing actual people who had attempted to do this. Folks from Citrix, Salesforce, AOL, Liquidnet and Wireless Generation all lent their opinions and experiences into our thinking. These were companies who were actively practicing Agile with their entire teams and, in most case, were far larger than us. If they were doing it with some measure of success, there was no reason little-old-us couldn’t, too.

Finally, we brought in Allistair Cockburn. His name is on the Agile Manifesto! If anyone had answers to our questions, it was him. While he did provide some insight, the questions he asked us revealed larger organizational issues that clouded the relatively focused task of integrating Agile and UX. Ultimately, Allistair’s visit helped us determine effective starting points for these efforts.

All of this research gave us what felt like enough raw material to put together a plan.

Our first attempt: just get it all done in 2 weeks.

What do you get when you take a process that used to take 9 months and try to cram it into 2 weeks? The short answer is frustration.

We kept the same processes, hand-offs and deliverables in place while adding in the concept of a 2-week sprint and daily stand-up meetings. We made the project sizes smaller but were still working sequentially. We failed to integrate the most important aspect of Agile—philosophy. We went straight for the tactics without considering why we were doing them. Our thinking didn’t change and hence there was no improvement in our collaboration or communication. The teams remained siloed in thinking and behavior. All we’d actually done was make deadlines shorter.

We did away with functional specs (and the interaction designers rejoiced!) and used the story card to replace them. The cards lived on boards and very quickly these boards began carrying the weight of the now-outlawed specs.

One of our Scrum team’s iteration board showing story cards, prioritization, activities, resource allocation and progress in the sprint.

Task designations on the iteration board. Example: Uc = Use Case, Tc = Test Case, Ia = Information Architecture, etc.

Further close-up reveals avatars indicating who is working on each card. Check marks indicate which activities have been completed.

They functioned as requirements documents (story cards), project plans (prioritization), resource allocation (avatars on boards to show who’s working on what) and status indicators (pins, colors, checkmarks, etc). The scope of supporting all of these elements quickly outgrew what one or two boards could handle and the number of boards multiplied (some would say to a ridiculous amount).

Wireframes also began to pick up some of the heavy lifting specs had left behind. In an effort to ensure all interaction rules were documented (partially egged on by developers uncomfortable without that level of detail) the interaction designers created annotations on the wireframes themselves that were often dense enough to obscure the experience being described.

Example of wireframe where the annotations were so heavy, the experience being depicted began to get obscured.

In an effort to keep track of the big picture (and combat the small-scale visioning of creating slices), we created a high-level vision document (essentially a sitemap or workflow) to serve as the barometer against which each change was measured. It also allowed us to keep the context of the smaller changes we were making in mind (relative to the big picture).

Example of a Vision Document (essentially a site map) showing the project slices within a broader context.

This failed.

No one owned it initially and since the process was driven by the development team (they were the only ones with any kind of Agile experience) the UX team didn’t feel empowered to defend such a document. In addition, due to the shared service nature, there was no loyalty to a particular business line or project since the assigned UX team members were just the “designers du jour.” It is nearly impossible to own the holistic theme when you may be abandoning it at the end of the two week cycle.

All roads lead to pain ....

Despite all of this upfront research, preparation, conversation, and insight, our first attempt at integrating UX into Agile yielded the diagram below. This was created by the UX team after a UX-only retrospective where we detailed all the challenges we were facing in this new world. All paths in the flow lead to the center of the diagram reading “[agile] creates a NEGATIVE ENVIRONMENT that FOSTERS FAILURE and generates LOW MORALE.” It is safe to say our first attempt was a failure.

Our second attempt: introduce two secret weapons

If we were going to get things working smoothly, it seemed clear that we needed more time to do our work. To buy us this time, we created and implemented a suite-wide style guide. The purpose of the style guide was to:

• Define a set of re-usable components once.
• Create a centralized, easily-accessible asset library for designers and developers.
• Provide a living document which designers and developers could develop as the site matured and evolved.
• Reduce the time developers needed to create repetitive elements (forms, buttons, UI elements, etc.).
• Reduce the number of design cycles by allowing designers to focus on the core experience while relegating the repeated patterns to style guide assets.

With the style guide in place, the UX team no longer had to worry about designing, defining and defending standard UI patterns. Instead, we could take the precious time we had in each sprint and focus on the core interaction problems.

A fringe benefit of the style guide was that now, in essence, everyone (including developers) was a designer. It leveled the playing field for many projects by allowing those not versed in interaction design to use the pattern library and create acceptable outcomes. Also, by providing fully designed elements, the style guide allowed interaction designers who were not strong visual designers to create polished, final-design-level wireframes. Developers also could create experiences without input from the UX team.

This tactic did indeed buy us the time we needed, but for some members of the team, it was too high of a price to pay. Making everyone a designer devalued the skills and expertise the team was bringing to the table. The way we mitigated this concern was by placing the most complex interaction problems within the UX team and “outsourcing” the simple ones to resources outside the team.

To gain even more time, we moved quickly into prototyping our designs. The team had transitioned fully to Adobe’s Fireworks product because of its strong prototyping features. While the code we were creating was throw-away code, illustrating the experience by showin rather than telling allowed us to ditch the heavily-annotated wireframes, facilitate better estimation at sprint planning meetings and have something to compare the working code to when it was ready for user acceptance testing.

Verdict? Win times two!

Our third attempt: put everything in line

Right now, you may be thinking, “Wait! You forgot about usability testing!” In the past we’d wait until close to the end of the cycle to test with users. With Agile we had to do something more frequent and less formal. Based on repeated attempts we ended up with this formula:

• Bring in no more than 3 participants each time.
• Test every other week (in a 2-week sprint situation) on the same day at the same time.
• Show the participants whateveris ready (this includes paper sketches all the way to working code).
• Schedule the session midway in the sprint leaving enough time to react to the findings.
• Invite EVERYONE to this standing session (don’t be surprised when they actually show up).
• Use the testing to “clear the boulders” out of the interaction (after 3 participants, it is clear what they are).
• Use the remaining time in the sprint to iterate on the design and validate with users again, two weeks later.

This has proven very successful. Recruiting for the testing is done via an external vendor, but everything else is run on site. Developers, product managers and executives regularly drop in to view the sessions (which buys implicit approval for the design tweaks UX makes), and the quality of the product at the end of each iteration is improved.

Speaking of approvals, we had to build those in to this new process as well. In the past, drive-by and email reviews were the strategies of choice. With the time-boxed nature of sprints, that approach just didn’t work.

We implemented two design reviews per iteration. The initial review is held midway through the sprint and serves to align the execution team with their product owners and project sponsors on the general direction of the proposed experience. The second review, scheduled two days before the end of the sprint, is meant as a final review. The design has to be 95% of the way “there” before the work could proceed to development. If it is not agreed to be 95% done, the project is pushed out another sprint, and the UX team spends another two weeks refining the design. In between the first and second designs, ad hoc reviews could be held if the designer felt there was a need for more fine-grain alignment.

For the design review meetings to fulfill their goals, attendance is critical. We hold design reviews at the same time on the same day every week, and the reviews are mandatory for all stakeholders. We also made sure everyone understood—and agreed—that missing a review meant implicit approval of the design. Attendance has been and continues to be strong.

Putting testing and design reviews in-line was a huge success for our teams. The design reviews provide designers with much-needed mileposts to strive toward while buying more UX design time through streamlined processes.

Verdict? Another win!

Our fourth attempt: bring everyone together and then separate

Collaboration was still missing from our scrum teams’ chemistry. Communication, we theorized, would come naturally if we could increase the level of collaboration. In addition, collaboration breeds alignment and a sense of ownership which in turn leads to less resistance and greater productivity.

We turned to the “design studio” technique for help.

Ripped off and bastardized from architecture school ,the design studio puts a cross-functional team together in one room and focuses on three activities:

• Sketch
  – within the confines of a forced timebox each participant has to sketch (literally, with a pencil and paper) a finite number of ideas on how to solve the given problem
• Present
  – then, each participant has to present each one of these ideas to the broader team and speak to its merits
• Critique
  – the other participants in the room proceed to critique each presentation based on merit and problem-solving efficacy (not on its fine art qualities)

This process is repeated three times, with each round increasing the fidelity of each sketch. Finally, one large sketch is presented with the highest amount of detail possible within the timeframe.

The benefits of this technique are manifold, but the most critical one is a sense of ownership and alignment from the cross-functional team. After participating in this session, everyone will see their “fingerprints” in the designs created later by the UX team. This feeling of ownership brings greater team alignment, as everyone understands the reasoning behind the design. In addition, should criticism be leveled at the design, the UX team has cross-functional support for the approach.

In addition, the UX team leaves the session with dozens of raw ideas to work through, evolve and incorporate into a final experience. This is much more productive than starting from a blank canvas.

Bringing together the cross-functional team to collaborate made one thing very obvious to us—the rest of the disciplines had dedicated resources to each project, while the UX team was still a mercenary squad going wherever the need was greatest. It became clear that we had to dedicate UX practitioners to each of our projects.

This change, while risking burnout on specific types of work, has proven tremendously successful. Camaraderie, communication and collaboration with UX—which never would have happened in our shared services environment—now flourishes and thrives. Spending time with the same folks every day working on the same problems binds teams together. Bonding breeds trust, and trust is core to Agile success.

Where we are today: habits are evolving. Slowly.

Years of training have taught designers to keep the kimono closed until the design is ready to be reviewed. This bought time AND control. Under Agile, designers need to be more open about what they’re designing and why, and they must be ready to show work much earlier than before. This is a tough change for designers to make.

We haven’t fully bought in to all of these changes quite yet. The team still feels that our new way of working dilutes the work, rushes it, and reduces quality. In addition, as mentioned earlier, it reduces the team’s perceived uniqueness and value they bring as trained designers to the organization’s success.

Collaboration is also tough because design is inherently a hero-based discipline. Everybody wants to be the person who designed the iPod or the creative genius behind Mint.com’s oft-lauded UI. Design awards get handed out to individuals, not teams—especially if a lot of a designer’s career has been within agencies.

Agile, on the other hand, is distinctly anti-hero. It’s about the team—first and foremost. For UX designers to integrate into Agile teams, the hero dream has to be left behind.

Another pivot point in the process where UX is currently struggling is the decision of what is a minimally viable product versus a minimally desirable product. We’ve been struggling with who defines what we release, when we release and what role UX plays in that decision. Currently, development decides in some situations, product management in others, the business in yet others. Our UX team needs to assert itself and influence these decisions to defend the brand and experience of the company. TheLadders is not a nascent startup that can afford to risk its existing brand awareness and values with the release of minimally viable feature sets. These feature sets can be light but must adhere to (or exceed) the experience to which our paying member base has grown accustomed.

Finally, in an effort to move beyond the staggered sprint model, we’ve begun experimenting with parallel pathing design and development. In these scenarios, UX and development start at the same time with the same end date. Designers are paired closely with developers and, instead of reviewing design deliverables, actual working code is reviewed each week. This tight collaboration reduces many of the dependencies described above but requires much more flexibility on the part of the designers, developers and most importantly business and product owners.

Our first attempt at this failed since we attempted to hold design reviews for UX work while development was ongoing. The reviews changed the path of the experience dramatically enough to warrant a pause in the work development was doing. Our next few efforts aim to subvert this process by reviewing working code—which has solid UX input. These efforts have been more successful due to the team’s comfort and trust working through staggered sprints together. We’re holding code/design reviews every two days on these efforts to review the experience, as it will appear to our users, and provide feedback on that experience (code, design, ux) – not an approximation of it as in the past.

Conclusion (for now)

We dove into Agile with hardly any knowledge. We’ve learned a lot through failure and iteration. The most salient learning here is not one of process but of philosophy. In order to truly become more agile (lower case on purpose) we must change the way we think about User Experience Design within the context of product development. If we can move away from our ingrained hero-based mentality and embrace more collaborative, open and shared product development models we’ll all be more successful—and ultimately, so will the businesses we support. The first step is communication, but the ultimate goal is trust.

The first step is communication, but the ultimate goal is trust.

This can’t be achieved out of the gate—it is the fruit of repeated tests and trials with your team through which you grow, bond and evolve into a high-performance, highly agile team.