Transformation is the act of taking a set of values from a dataset, processing them in some way (depending on the aims of the research) and arriving at a new set of values with the goal of revealing some aspect of the data from a new perspective.

(This article is the fourth part in the Deconstructing Analysis Techniques series.)

This technique is characterised by the fact that the values are changed; that someone looking at the new values will be unable to work backwards to the original values; and that for each original data point there is a single, new data point.

In mathematical parlance, (and you can skip this part if you like) the difference between a manipulation technique and a transformation technique is that manipulated data sets are congruent with the original, whereas transformed data only maintains cardinality (i.e. the same number of elements).

So, what does that all mean? We’re talk here about analysis methods like:

• Scaling – taking one set of data and massaging them to fit a distribution or ‘shape’ of values.
• Moving averages – taking a number of consecutive values and averaging them as way of ‘smoothing’ the last value in the series.
• Weighted averages – calculate an average value where more importance – ‘weight’ – is given to some values.
• Weighted indexes – calculate an indexed score (against a baseline) where more importance – ‘weight’ is given to some values.
• Seasonal adjustments – an adjustment made to a data point to account for cyclical peaks and troughs to highlight the ‘real’ shift
• Differences – a method of looking at the changes between one value and the next.

Now, initially, most of these methods may feel pretty technical, quantitative and removed from standard design research analysis. However, they form a powerful collection of analysis methods that will better equip you in undertaking design research. They also represent fairly low-level mathematical/quantitative methods and are available in a standard spreadsheet program. More importantly, used properly, these methods – and transformation techniques generally – open up new avenues for understanding the people who will use the services and products we design.

Used properly, these methods – and transformation techniques generally – open up new avenues for understanding the people who will use the services and products we design.”

Scaling

In “Deconstructing Analysis Techniques” we used the example of fitting test scores to a pre-determined probability distribution – scaling – as the example for Transformation techniques.

When we measure a population characteristic – such as height, or a test score – we create a sample set of data for that characteristic (unless we are measuring the entire population). There are times when the raw distribution (the frequency of occurrence for each value in our data) of results is not what we’re after. We may wish to compare the shape and attributes of two separate samples – two groups of test participants, for example – and so we transform the two sets of data so that they share a common mean (the average value for the data set).

Usually this is done to bring both sets of data to what is known as a ‘normalized’ distribution with a mean of 0. Of course, in our test/exam result example, we want to adjust the scores so that the class as a whole receives a pre-determined number of A, B, C, D & F. What we’re doing here is to adjust the overall shape of the data. (In these cases a plot of the raw data will look different to the scaled data.) When graphed the scaled data will look roughly bell-shaped, with the middle – or ‘hump’ – representing average performance, and the two thin tails representing high-performance (at the top end) and failure (at the bottom end).

Moving Averages

A moving average is used to smooth out day-to-day fluctuations with time series data. It is, literally, the average of the previous x days’ worth of data. A good example would be the number of page views received by a site. Each day the data will jump up and down, creating a sense of “noise” that makes analysis difficult, and, when a small number of observations are looked at in isolation, can create a false impression. A moving average is useful in time-series or longitudinal studies where we measure the value of a characteristic for a single object (person, server, site etc) over time.

One rather well-publicised and important example of this is the series of global temperature readings that have been used by both sides of the climate change debate. Skeptics of global warming point to a recent period of observations (2002 – 2007)   which show a decline in global average temperatures. When the same data is looked at using a moving average, smoothing out the peaks and troughs, a clear upward movement is seen.

The choice of time period to use when calculating a moving average is based on the specific circumstances of the data. However, common sense is usually all that’s required. For example, when looking at Web traffic, a moving average calculated over 7 days is sufficient to counter spikes that occur during a given week. You might also calculate a moving average over a month if fluctuations occur over a longer cycle.

Weighted Average

Weighted averages aim to address one of the criticisms of a moving average – and other types of averages – that being all values in the average are treated equally. It is often the case that one observation is more significant or important that another.

Let’s say for example we’re measuring the time to complete a task in a user evaluation session. We have representatives from each of our personas (or other audience segments): 2 primary personas, 3 secondary personas, and one tertiary persona. In this case, the performance of the two primary persona representatives is far more significant than that of the tertiary participant.

When we calculate the mean time-to-complete value, we can weight the results so as to reflect the relative importance of each participant. We may assign (and the exact values will vary for you) a weighting as follows:
Primary: multiply by 9
Secondary: multiply by 3
Tertiary: no multiplier

What we’re essentially saying is that our secondary personas are three times more important than our tertiary persona; and that our primary persona are three times more important than our secondary. We could just as easily use a factor of 2 (instead of 3) leading to values of 4, 2 & 1 in the example above; what matters is that we use weighted averages to adjust the dataset to account for the relative importance of some measurable data set by some exogenous variable.

Weighted Index

An indexed value is one measured in terms of some baseline figure. The aim is to convey movement around a starting point when there is no way to specify a zero.

An example of an index might be a satisfaction score. Since satisfaction a largely subjective measure, there is no way to define a zero point. Instead we typically measure a ‘pre’ figure and map that over time. Common values for an index are zero and 100. The choice is arbitrary and is typically chosen for clarity in communication.

Indexes are often calculated as an aggregate of a number of measurements. But it is also the case that we sometimes need to treat the data we receive from one group as being more important than another. This is where a weighted index comes in handy. A weighted index  – like our weighted average – treats different values as more or less important.

So, if it is common practice to design a product or service to better meet the needs of our primary audience segments; it also makes sense for our satisfaction index to put more stock in the satisfaction of our primary segments. We do this by applying a weighting (some multiplier) to each piece of data collected based on its relative importance.

We could easily do the same with responses to a question like “Would you recommend this service to a friend?”

This technique provides us a with a convenient way to build positive bias – towards the needs of our important audience segments – directly into our research methods.

Seasonal Adjustments

Some of the things we observe in design research are subject to cyclical variations. We may not, however, want to include a change in our data due to “seasonal” fluctuations, instead wanting to identify “real” changes (in frequency of use, for example).

In order to look at the real changes in our observed data we need to account for the seasonal variability first.

A familiar example might be to look at the number of page views or unique visits received by a site. We might see a big lift in traffic between Sunday & Monday; and a big drop between Friday & Saturday. In order to tell whether an observed drop in traffic on some Saturday is “normal”, we need to look at the regular pattern of changes and “adjust” the Saturday figure.

One way to do this is to calculate the average drop in traffic over time (between Friday & Saturday) and then apply this to the current observation for Friday. This as a predictor or estimator for the current Saturday, which we can then compare against the actual observed data. The average difference acts as our seasonal adjustment.

The Adaptive Path Aurora concept uses a scenario where a farmer shows that their farm will still have rain, using seasonal adjustment. See video

Differences

There are times when what we’re interested in knowing is not the raw value of an observation but the change between one observation and the next.

The calculation (transformation) is simple: for each pair of observations, subtract one from the other. Of more interest is why we would want to know such a thing.

Consider a test of a new design in which we test first the time to complete a task with the current design; and then the same task with a new design. Across all participants in the test the raw observations (i.e. time to complete) is far less interesting than the change in that time as a result of the new design. (Note that we may wish to express that change as a percentage rather than a raw value.)

We can use the same technique to highlight the variability of some observation over time. For example, we may be tracking the number of connections or ‘friends’ a person has in some social network to understand the relationship between the current number of connections and the rate at which new connection requests come in. To identify the number of new connection we simply calculate the difference between successive observations.

Summary

Although primarily applied to quantitative data, transformation techniques are useful in a wide range of design research activities beyond the quantitative.

Transformation of our research data can act as a way of reducing noise and bringing into sharp relief characteristics of the underlying user behaviour. The act of transforming removes us from the raw, original data, but in doing so we can gain the opportunity to uncover meaningful insights hidden from us otherwise.

The ability to “play with the data” is a critical capability in analysis. We utilize this technique in many situations: searching for patterns or trends in our observations; or as another preparatory stage for further analysis. Sorting data in some way – alphabetic, chronological, complexity or numerical – is a form of manipulation.

(This article is the third part in the Deconstructing Analysis Techniques series.)

Manipulating data is that process of re-sorting, rearranging and otherwise moving your research data, without fundamentally changing it. This is used both as a preparatory technique – i.e. as a precursor to some other activity – or as a means of exploring the data as an analytic tool in its own right.

One of the key characteristics of a manipulation technique versus related techniques like transformation is that the underlying data remains unchanged. The main thing we’re doing is changing the relationship – logical or physical – that one piece of data has with another.

Reorganizing the data helps us to identify patterns that may otherwise not be apparent. In fact, it is almost certain that most patterns won’t be visible at first glance.

Let’s start by taking a more detailed look at some of the processes that contribute to the manipulation of data.

Re-sorting is literally a technique aimed at changing the order of the data. Re-sorting is most often carried out on numerical or quantitative data, but can just as easily be applied to text content. There are a few common types of sorting – numerical, alphabetical, chronological; as well as some that are much less common. For example, a list of responses to a survey question asking for a rating of a service might be sorted based on the severity and tone (positive or negative) of the review.

Sorting data helps to isolate significant individual values – the highest or lowest, most-frequent or least-frequent, first or last; and can also be a way of highlighting the shape of the data (more on this later).

Re-arranging is an activity that typically involves the physical or digital repositioning of a data element so that it sits in closer proximity to another. This might be to organize photographs into a narrative; or to juxtapose contrasting ideas for discussion.

Rearranging ideas through the manipulation of Post-its. Photo courtesy of Todd Warfel.

Much of the rearranging we do is exploratory, although at times it will be more directed. In these cases we might be trying to present a new configuration for our data – like rearranging furniture – to better support some activity.

Some of this manipulation will be more purposeful. We might be seeking to categorize a collection of photographs by grouping them into similar piles; or draw out common themes in user interviews. Recall, for example, in our article on Deconstruction we talked about breaking out key phrases or ideas into separate data points (on index cards, post-it notes etc).

What are we trying to achieve, though, with all this moving about?

Patterns

I’ve written previously on the important role pattens play in analysis; and the different types of patterns one might seek to find and identify in research data. The patterns we seek include:

• Trends: the gradual, general progression of data up or down;
• Repetitions: a series of values that repeat themselves;
• Cycles: a regularly recurring series of data;
• Feedback systems: a cycle that gets progressively bigger or smaller because of some influence;
• Clusters: a concentration of data or objects in one small area;
• Pathways: a sequential pattern of data;
• Gaps: an area devoid of observations;
• Exponential growth: rapidly increasing rate of growth;
• Diminishing returns: there is a decreasing rate of growth;
• Long tail: a pattern that rises steeply at the start, falls sharply, and then levels off over a large range of values.

Immersion

Many design researchers and design practitioners talk about the need to immerse themselves in the data before they can make any kind of sense of it. Manipulating the data is a way of gaining that immersion – that familiarity – through direct engagement.

Designers will undertake this process in a number of ways, depending on the format in which data has been stored. One of the most popular forms of manipulation is to write out key concepts, observations, and ideas onto Post-It notes and stick these to a wall.

The design team them actively moves the physical Post-It notes around, rearranging and grouping concepts and observations to help trigger creative ideas. This technique may be used in both the analysis and design processes to assist the design team, and there isn’t a write or wrong time at which it can be undertaken. This type of exploratory analysis can be powerful, and is a key tool in the card sorting analysis arsenal.

If running the card sort was the fun part, analysis is the painful part, at least until you get going. Exploratory analysis is like playing in the data – looking for connections that make you think “hey, that’s interesting”, or that show patterns of behaviour. – Donna Spencer, Card Sorting

Donna’s quote highlights two important characteristics of this analysis technique: firstly, that it can help uncover and highlight key insights in the design research data; and secondly, that sometimes starting is the hardest part.

Where Do I Begin?

Design research – any research activity, really – can result in a body of data that simply feels overwhelming. Thousands of sticky notes containing observations or notes, covering the walls of a ‘war room’. Perhaps it’s thousands of survey responses, or dozens of interview transcripts. It may be hundreds of photographs taken of users in context; or hours of video of a user testing study.

Sometimes this richness of available data works against us, making it difficult to understand where we should begin. Like it’s counter-part in analysis – Deconstruction – the techniques of Manipulation are easy to undertake, and require little or no preparation.

Perhaps more importantly, Manipulation encourages exploration. It works well as an unstructured activity and therefore works well as an entry point into those vast collections of messy data points we’re so often faced with early in the analysis. If you’re not sure where to begin, begin with manipulation – the more tangible and tactile the better.

Uses of Manipulation

Despite the simplicity of manipulation as a technique, it delivers the heart of some very powerful analytic methods. For example, affinity diagramming is requires little more than manipulation (and perhaps deconstruction as a preparatory technique) to produce some real insights.

In many respects, the method of creating a mental model introduced by Indi Young in her book of the same name is another example of manipulating data with intent. Throughout the method data is manipulated – usually physically – through the use of sticky notes or index cards. Ideas are grouped and compared, collated or isolated, by physically repositioning and rearranging the physical object.

Manipulation can also be used to answer specific research questions. We can sort our data chronologically to find the first occurrence of an event. We can sort the data numerically to identify the highest or lowest values, or to identify the median figure (the middle observation) in a series of observations.

Perhaps we’ve already gone through an exercise of aggregating data points and tallying up the number of occurrences of each. We can now manipulate the data and sort in either ascending or descending order to identify the most common or least common responses. This combination of techniques – aggregation and manipulation – provides for an unsophisticated, but still useful ‘method’.

And that, of course, is one of the key things about each of the analysis techniques discussed in this series: whilst each is useful on its own, their real power comes from the ways in which they are combined to form the sophisticated and rich methods we tend to encounter in books.

Challenges

One of the greatest challenges we face when we start to play with our research data is a tendency to settle on the first arrangement; the first patterns; the first grouping. We begin with such a chaotic mess, that first glimpse of something that presents us with a clear view – some sense of real meaning – can be quite powerful. We resist the step of re-shuffling and messing it up again, and may therefore miss the opportunity to see a second, third or fourth pattern.

Another major challenge – which we’ve mentioned above – is that the volume of data can be quite daunting. As much as it is a good step to just get started, some sense of how data elements can be grouped or arranged before you begin is important; but it shouldn’t be an obstacle.

And, of course, we must be in a position to easily manipulate the data we’ve collected. This means the format and medium in which our data is recorded is critical. Storing data digitally is not necessarily advantageous or preferable: many designers will attest to the positive effects that can come from physically interacting with the data.

Summary

Manipulation can therefore be seen as one of many low level analysis techniques with which we work every day. We’ve all encountered it in one form or another, and probably spent little time considering it. And yet it is one of the major workhorses of any analysis effort, and one which we should understand.

Deconstruction is one of the most frequently used and fundamental analysis techniques in our toolkit. It is used as both a preparatory technique to get research data ready for use in other ways; and a powerful technique in its own right as a method of isolating, exposing, and testing assumptions deeply embedded in our mental models.

One example of deconstruction is turning an interview transcript into a series of separate comments or answers to questions. Deconstruction is often used simply to prepare data for other analytic processes such as manipulation or summarization, or even abstraction.

Note: this article builds on the first part of the series: Deconstructing Analysis Techniques published in February.

Examples of Deconstruction

• Chemical analysis – mass spectrometry: a technique for determining the elemental makeup of a substance or molecule.
• Philosophy/literary criticism: a technique of isolating and testing ideas contained within a work of philosophy or literature
• Systems analysis: identifying root causes through the identification of individual system ‘actors’ and their interactions
• Quality control: unit testing functional components of an application, requires first identifying those components (typically by recourse to the specification)
• User interviews: identifying individual concepts or ideas
• Card-sorting: working with card-pairs
• Task analysis: breaking down complex activities into individual tasks and their components.

There are a wide range of examples of the way in which deconstruction occurs, but our aim is always to reach a definite ‘atomic’ state (where the atom is defined by our research objectives). It should be noted that there are typically more things going on than merely breaking down the data. In the case of chemical analysis, level of elements or compounds are measured; in the case of a stakeholder or user interview, the individual words, phrases or ideas may be tallied, grouped, manipulated or otherwise worked with to form some new insight.

Deconstruction can – and often is – built into the design of the research. We see this in online card-sorting, for example, where data is stored from the outset as card-pairs. Survey results are another example of data where pre-deconstruction is built into the research.

Why, though, this urge to break data down into smaller and smaller pieces?

Smaller, more granular data provides for greater flexibility in the other analysis techniques we need to undertake. By separating ideas or objects out into their own data elements we can have greater control over how elements are treated and positioned with respect to other elements.

For example, splitting a Name element into separate First Name and Surname elements allows us to treat these two components independently, and ask a broader range of questions – such as: “What are the most common first names?”

Smaller, more granular data provides for greater flexibility in the other analysis techniques we need to undertake.

It requires extra effort to break data down and then to store it in more granular form. It also takes effort to request and record extra data during the research process itself. So, whatever level of data granularity we use should be for specific reasons, and to address specific research questions.

Deconstruction represents a powerful analytic technique in its own right. By isolating concepts and ideas, and exposing them to scrutiny on their own, deconstruction highlights the existence of untested assumptions and ‘sacred cows’.

In this sense, deconstruction is often used to analyze problems or situations to which we need to formulate a response. This use of deconstruction allows us to test the reality of perceived constraints: by isolating each constraint to the design, and looking at the conditions under which they may hold true, new possibilities can open up that may otherwise not have been possible or feasible.

Dangers in Deconstruction

There are dangers in deconstruction that are worth mentioning here. At the end of the day our work should lead to something substantively new. This can be difficult if we lose sight of the macro-level problem in pursuit of an understanding of the data in finer and finer detail.

Secondly, in studying the fine detail of our data we can miss seeing the patterns in our data that help drive insights and accelerate the transition to design concepts. At the same time, some patterns only become visible or apparent when we reach a level of granularity appropriate for the data.

Deconstruction can also generate noise in our data which obscures our sense-making abilities. This noise may be the result of data overload – simply having too much information to allow for processing; or it may be that small-scale, natural random variations are masking higher-level trends or patterns. In these cases, the use of summation and aggregation techniques might be an appropriate contrast to the deconstruction technique.

Deconstruction in practice

Deconstruction can often be used in very close association with other analytic techniques. For example, we may break data down into more granular form to facilitate manipulation of that data as a means of inspection or ‘eye-balling’.

In the same vein, we may manipulate or transform our data to allow us to zero in on a particular characteristic – deconstruction in the critique sense of the term.

It may help at this point to look at some examples to help illustrate the different uses of deconstruction as an analytic technique:

User Interviews
A typical interview scenario involves asking participants a series of questions (usually open-ended; sometimes based around topics rather than using a strict question set) and recording the responses. Recording may be through the use of written notes, audio recording, video taping, interviewer/observer notes; and may include a combination of the above.

To begin drawing connections and identifying themes between interviews we need to break down – or deconstruct – the interviews to the level of individual ideas or concepts, feelings, thoughts etc. The medium we use to record each of these ‘objects’ is not important: a spreadsheet might be used just as effectively as Post-It notes or index cards.

Once the data is in this more granular form we can carry out further analysis on the interviews. We may, for example, want to look at the prevalence of positive versus negative feedback.

Note, however, that the need for deconstruction is entirely dependent on the questions we are trying to answer through our research. For example, if our intent was to formulate an impression of the overall level of satisfaction for each interview subject, the deconstruction would be an entirely unnecessary task.

Diagnosing Causes
When faced with a failure in a complex system – such as the inability of users to complete a multi-step process, or the appearance of a previously unplanned-for edge case – it is typically quite difficult to diagnose the cause of the failure (in the absence of error handling designed specifically with this in mind). In order to identify the root cause of the failure we undertake a deconstruction exercise to help isolate the components of the systems.

Designing a Car: Highlighting untested assumptions
If we were to begin designing a car we might begin with a brain-storming session and list out all of the components or features that are needed. That list might include items such as “wheels”, “engine”, “fuel”, “doors”, “seats” and a whole range of others. We can now look at each of these features and ask why it’s there, and what it says about our notion of the solution.

For example, ‘fuel’ presupposes a form of combustion engine which, increasingly, may not be relevant. More importantly, ‘fuel’ highlights a range of assumptions – mostly tacit – derived from our mental model of the object ‘car’.

Once these assumptions are exposed we can begin to question their validity in the context of the problem – instead of pre-defining a solution in the statement of the problem. Such questioning, enabled through deconstruction, opens up a broader perspective on the design of a solution.

Conclusion

Deconstruction serves a dual role in our analysis work: as both a preparatory technique to get research data ready for use in other ways; and a method of isolating, exposing, and testing assumptions deeply embedded in our mental models.

The technique is not without its drawbacks: more granular data requires effort to gather and record, store, and analyze. It can also generate ‘noise’ in the data, which can obscure instead of illuminate.

Understanding the role of deconstruction in analysis can help us to better target it’s application to the solution of specific research questions.

Photos by s1mone (card sorting), andercismo (magnifying glass), smiling da vinci (interview)

Analysis is that oft-glossed over, but extremely important step in the research process that sits between observation (data gathering) and our design insights or recommendations. In many respects, analysis is crucial to realizing the value of our research since good analysis can salvage something from bad research, but the converse is not so true. This is where the literature tends to fall a little silent, jumping over the analysis techniques straight to a discussion of how best to document and communicate the findings from analysis. This article seeks to begin to redress that imbalance by breaking down the analysis black box into its major sub-techniques.

On a recent project I needed to collect and analyze the content management templates in use across a large enterprise Intranet. We were looking to inventory the diversity of templates in use; whether they existed outside or within the enterprise content management system; what changes might be made to the ‘official’ template set to reduce the overall number of templates, and to prepare for the migration of all content to a new design a few months down the track. I looked around at the literature for information architecture and Web design generally and found quite a few references to content inventories and content analysis, but nothing on analyzing templates.

I set about designing the analysis task from scratch: looking at what we wanted to get out of the analysis; and looking at what tools and techniques would most effectively allow us to get there. In so doing, it struck me that there is very little information published about the process of analysis that would equip practitioners with a toolkit to construct their own analytical techniques. So User Experience literature and all of its component domains focuses on techniques for user research and testing, it’s surprising to realize that the coverage often skips over the process of analysis, since this is where much of the value of our research is realized.

Techniques of Analysis

We can start to pull back the curtain on analysis by looking at the techniques that go into the process:

• Deconstruction: breaking observations down into component pieces. This is the classical definition of analysis.
• Manipulation: re-sorting, rearranging and otherwise moving your research data, without fundamentally changing it. This is used both as a preparatory technique – i.e. as a precursor to some other activity – or as a means of exploring the data as an analytic tool in its own right.
• Transformation: Processing the data to arrive at some new representation of the observations. Unlike manipulation, transformation has the effect of changing the data.
• Summarization: collating similar observations together and treating them collectively. This is a standard technique in many quantitative analysis methods.
• Aggregation: closely related to summarization, this technique draws together data from multiple sources. Such collections typically represent a “higher-level” view made up from the underlying individual data sets. Aggregate data is used frequently in quantitative analysis.
• Generalization: taking specific data from our observations and creating general statements or rules.
• Abstraction: the process of stripping out the particulars – information that relates to a specific example – so that more general characteristics come to the fore.
• Synthesis: The process of drawing together concepts, ideas, objects and other qualitative data in new configurations, or to create something entirely new.

Let’s take a look at each of these techniques in detail and discuss some of the ways in which each technique can be applied.

Deconstruction

Breaking observations down into component pieces. This is the classical definition of analysis.

Breaking down research data into its component parts is a standard technique for analysis. One example of deconstruction is turning an interview transcript into a series of separate comments or answers to questions. Deconstruction is often used simply to prepare data for other analytic processes such as manipulation or summarization, or even abstraction.

The aim of deconstruction is to decouple each component so as to allow inspection of each in its own right. In other disciplines this process is used as a device for critical thinking, bypassing the potentially misleading image conveyed by the whole. In so doing deconstruction can be a powerful tool for exposing unquestioned assumptions about our users’ mental models or the business priorities of the client organization.

Looking at our template analysis example, one of our first analysis tasks was to deconstruct the templates into their components. Like most of the technique we took a very low-tech approach to the task, blocking out the individual components with a pencil. In our case, the deconstruction made easier a lot of the subsequent analysis work.It was a minor, but significant, step in the overall process.

Manipulation

Re-sorting, rearranging and otherwise moving your research data, without fundamentally changing it. This is used both as a preparatory technique or as a means of exploring the data as an analytic tool in its own right.

The ability to “play with the data” is a critical capability in analysis. We utilize this technique in many situations: searching for patterns or trends in our observations; or as another preparatory stage for further analysis. For example, sorting data in some way – alphabetic, chronological, complexity or numerical – is an a form of manipulation.

The ability to easily manipulate data is one of the key determinants for the tools we use in our analysis work. Spreadsheets are an excellent tool for manipulating data; but as we see in our template analysis task, the use of a more tangible form – such as our index cards – can be just as effective: if not more so in some cases.

When data recorded in a format that resists fluid manipulation and exploration people can stumble when moving from observation & data collection into analysis. It is important to plan this task into the research design so that it is not overlooked. You could find yourself with a costly and time-consuming data-entry process  if it is forgotten in the planning stages.

Transformation

Processing the data to arrive at some new representation of the observations. Unlike manipulation, transformation has the effect of changing the data.

Transforming research data is the process of taking our research data and turning it into something else. For example, you may recall from your schooling days the practice of “scaling” results from an assessment task (exam, essay etc) so they fit a certain distribution, so you end up with (for example) 10% A, 15% B, 25% C, 25% D etc

Another example might be to convert raw data into a logarithmic form to reduce the impact of extreme values – or to demonstrate power laws in the data.

Summarization

Collating similar observations together and treating them collectively. This is a standard technique in many quantitative analysis methods.

The goal of summarizing data is to generate an additional set of data, typically more succinct, that encapsulates the raw data in some way. This may be a short sentence that captures the essential point from several minutes of an interview transcript: “participant finds site search unwieldy, confusing and difficult to use”.

We can also summarize the data quantitatively using summary or descriptive statistics such as frequencies, means, and standard deviations. Unlike the process of abstraction, where specificity is sacrificed for the sake of clarity; or aggregation, where several data sets are “rolled up”; summarization seeks to characterize the underlying data.

Once again, spreadsheets are a very useful tool, especially when dealing with quantitative data. But they can be similarly useful when handling other data types. An equally useful medium for capturing summaries (once you have them) – particularly of qualitative data – is the PostIt or sticky note. This medium is also highly suited to manipulation and exploration of the resulting data. One advantage sticky notes have over a spreadsheet is that you can arrange and re-arrange them in two dimensions, so you can further manipulate and explore the summaries.

Index cards share many of the same advantages as sticky notes. They can be an excellent tool for capturing and working with summaries. They have the added advantage of being relatively robust and can therefore sustain a greater degree of handling.

Aggregation

Closely related to summarization, this technique draws together data from multiple sources. Such collections typically represent a “higher-level” view made up from the underlying individual data sets. Aggregate data is used frequently in quantitative analysis.

As discussed previously, aggregation is similar to, but distinct from summarization. In one respect aggregation is simply the process of bringing together data from a variety of sources and adding it together. In an analytic context it also carries with it the connotation of combining those sources together into something new.

A good example to highlight aggregation in action is the creation of a (fictional) customer satisfaction index (CSI). Our CSI will use data from:

• An annual customer survey;
• The number of product returns received; and
• The ratio of new to repeat customers.

We combine data from each of these sources and arrive at some single figure – based on some form of calculation (we’ll save the ‘how’ of that for another time). That single figure – which we can track year-to-year – is our aggregate. Unlike a summary, which characterizes a single piece of data, you can see that our aggregate is a composite value.

Generalization

Taking specific data from our observations and creating general statements or rules.

Taking the results of some specific research task and drawing general inferences about the broader population is one of the most common, but perhaps the least understood analytical technique. Generalization draws a great deal of its strength from the discipline of statistics, and the particular techniques of statistical inference.

In many respects generalization is similar to abstraction in that it reflects a move from the specific to the general or essential. It is a way of describing the common characteristics of the objects reflected in the data.

An example of generalization might be: “security is important to our users” based on an analysis of user interviews.

Abstraction

The process of stripping out the particulars – information that relates to a specific example – so that more general characteristics come to the fore.

The process of abstraction involves the progressive removal of specific data retaining just the essential information needed to communicate particular characteristics of an object. For example, “professional” is a more abstract form of “Doctor” or “Lawyer”; “graphic” is a more abstract form for “photograph”, “logo”, “illustration” or “chart”.

A wireframe is an abstract representation of a page design; the template thumbnails on our index cards are an abstract representation of the templates.

Abstract representations can be very useful because they remove a lot of visual noise from the analysis process. What we’re left with is a “high-level” depiction devoid of specific detail; highlight focused on just those elements which are relevant to the discussion.

Synthesis

The process of drawing together concepts, ideas, objects and other qualitative data in new configurations, or to create something entirely new.

Combining multiple elements together to create a new, complex ‘thing’ is what the technique of synthesis is all about. Similar in some respects to aggregation, synthesis typically deals with non-numeric data.

Synthesis is often undertaken towards the end of an analytic process as the reverse of deconstruction. So where we might begin by breaking down data into its component parts and examining them; we often end by recombining those components in new ways. Note, however, that synthesis can also form part of an exploration and is one of the fundamental tools of the trade for UX strategy work.

If deconstruction allows us to critically examine assumptions by isolating individual components, synthesis allows us to explore new configurations for the whole.

But what about…

In discussing this article with other people we identified three other techniques that we either weren’t sure belonged as analytic techniques, or we couldn’t decide if they were already covered by the techniques discussed above. We believe they’re all very important to the analysis process. They are:

• Reflection: thinking, pondering, contemplating. To the outside observer it looks a lot like staring into space, but your mind is going over and over and over all the detail of your observations, data, diagrams, and other research materials. It’s the part you can’t put a time limit on, and can make or break your subsequent work. You might call it “soaking it all in”, or “immersing myself in the data”. This technique is incredibly valuable to me in my own work and I’m not sure I’d be as effective if I didn’t include it.
• Visualization: this technique is about giving the data a visual dimension. Instead of lists of items, or rows of numbers in a spreadsheet, a chart or graph or some form of illustration. A good visualization can help expose patterns or gaps much more clearly than the raw data.
• ‘Number-crunching’: this feels like it needs to be drawn out as a separate activity from data manipulation, transformation, or summarization, but I also recognise that this level of distinction may just be peculiar to me. This refers to all of the heavy-duty quantitative analysis work like clustering analysis, or regression, calculating correlation co-efficients and the like.

Conclusion

Working with research data and observations is often treated as a black box in design literature. Designers find themselves faced with the daunting task of analysing research data, but lack clear approaches to that task. Understanding the major techniques used in analysis work can remove some of the uncertainty and provide a clear way in to the work.

There still exists a very large gap in the literature on analysis and analytic techniques, but I hope that this discussion of the major components of analysis will go some way towards filling that void. The next time you’re undertaking some analysis work, try and identify these major techniques, and see if there are any others we can add to the list.

I’d like to say a very big thank you to the people who helped clarify and refine both my thinking on this topic, and the expression of that thinking in this article: Will Evans, Livia Labate, Donna Spencer and Daniel Szuc; Christian Crumlish, Michael Leis and Kaleem Khan.

Graphics by Jeroen van Geel (and he’s pretty proud of them .