In a world of unlimited access to information and infinite choices, it can be hard to make decisions. We’ve created tools to help us sort the humongous mass of information we’ve created; tools to help us find what we’re looking for. We’ve grown so used to using these tools that we rarely notice their downsides and limitations. This article aims to highlight some problems related to findability and discoverability and encourage you to find alternative solutions to the existing paradigms.

What is serendipity and why is it important?

Serendipity denotes the property of making fortunate discoveries while looking for something unrelated, or the occurrence of such a discovery during such a search.

The experience of browsing items in a physical space or online catalogue can differ substantially. For example when you’re browsing records in a store you often come across items you weren’t actively looking for but which you instantly recognize as desirable. Online stores offer some mechanisms for discovery but they’re highly limited in scope when compared to physical environments. They may offer a much higher number of items on sale than a physical store, but because screen space is scarce catalogues have to resort to categorisation trees, so users are only exposed to a small subset of the full range of possibilities. Online, users have less peripheral vision and a limited awareness.

“First we shape our tools, thereafter they shape us” – Marshall McLuhan

Existing Paradigms

So far we’ve had three paradigms for enabling findability: search, categorisation and social recommendation.

Search – the mirror

Search is great when users have a precise idea of what they’re looking for, but lacks the ability to propose related content that could bring new insights into the subject the user is researching. It acts as a mirror, you only get what you put into it. The problem is compounded by predictive text, which actively influences a user’s decision process while she’s typing a search query. The problem is the algorithms behind the search and autocompletion mechanisms are based on statistical data culled from the collective behavior of all users. This leads to the progressive erosion of the less used words from the suggestions mechanism and can have a huge impact on a user’s perception of a given subject. This can have terrible implications, for example consider how someone’s perception of a political event can be shaped or influenced by the keywords being suggested to them while they’re conducting searches. Would you be happy to have your worldview influenced by Google’s auto-complete?

Add to the equation that Google is a private company whose profits depend on advertising and sponsored results, it is a no-brainer to imagine that predictive search will soon start featuring sponsored suggestions too.

What is popular becomes exponentially popular, and everything else gets buried with no chance of surfacing. The price we pay for Google’s efficiency is having our perception of the rich complexity and variety of the world filtered and reduced to its average sum.

Category drill-down – a box inside a box inside a box

As a user traverses a navigation tree, her choices exclude a high number of paths that could provide unexpected insights. Of course users can trawl up and down the tree, and cross-referencing mitigates this limitation, but this becomes unpractical for very large collections. Cross-referencing is often limited in scope, what usually happens is you have a group of objects sharing some common theme pointing to each other but this forms a closed loop, so if you follow the cross-reference links you end up going full circle and sooner or later you find yourself looking at the initial item again.

Social recommendation – The echo chamber

Recommendation engines are the prevalent method for adding serendipity to navigation systems, however they operate in a rather linear fashion: they analyze a user’s past behavior and that of his peers, and use that information to extrapolate recommendations. The intended purpose of recommendation engines is to suggest to users objects or information which might interest them. But because the recommendations are based on a user’s past behavior, it tends to only offer back to the user items within their existing range of interests. So it reinforces a user’s tendency towards a certain behavior and never triggers alternative responses. Instead of offering something truly new to a user, recommendation creates a self-referential loop where a user’s body of knowledge is limited by her existing frame of references.

The notion of “Echo chamber” describes this process: “ [...] participants may find their own opinions constantly echoed back to them, and in doing so reinforce a certain sense of truth that resonates with individual belief systems.  This can create some significant challenges to critical discourse within an online medium.

the echo chamber

The echo-chamber effect may also impact a lack of recognition to large demographic changes in language and culture on the Internet if individuals only create, experience and navigate those online spaces that reinforce their “preferred” world view.  Another emerging term used to describe this “echoing” and homogenizing effect on the Internet within social communities is “cultural tribalism” [...] “ – Wikipedia (On a related note but from a journalism point of view Eli Pariser talks about what he calls the “filter bubble”, worth checking out.)

Mob Rulz

Because people spend so much time on social network platforms, it is safe to assume this will have an impact on culture. It impacts how we relate to those who don’t share our worldview, creating invisible walls between people rather than connecting them. The reverse side of connecting with some people on social networks is we are simultaneously isolating ourselves from all the other people we are not connected with. Users are implicitly creating two groups, those who belong to the circle and those who don’t. We create safe bubbles for ourselves where only the echo of our own preferences can be heard, and opposing views never get a chance to be heard. It’s like a child who is given the option to only ever eat what they please, and exclude anything they don’t know or like, as on Willy Wonka and the Chocolate factory. Seducing if you’re 5 years old perhaps, but would you really enjoy only eating snacks and candy for the rest of your life?

Alternatives

So what can we do to increase the serendipity of the systems we design? Being aware of the issues is already a good first step, I suspect what happens is designers specify a recommendation system but never get down to thinking about the details of how the algorithm behind actually works. So it’s up to the developers to decide and this usually defaults to the “top averages” approach. In most cases no-one even notices these funneling effects taking place.

Search – revisited

In the case of search there is no easy answer, it’s supposed to be functional and work as a mirror. What can help in some contexts is to offer alternative search terms to be combined with the initial query, or showing loosely related results clearly identified as such. Google has changed their results page while this article was being written and now it sometimes displays a “Something different” category as part of the faceted navigation for filtering results. For example if you type “Johnny” it assumes you’re looking for Johnny Depp and under “Something different” it suggests “george clooney, orlando bloom, matt damon, leonardo dicaprio, brad pitt”

It’s going through the most popular results, doing a semantic analysis that identifies “Johnny Depp” as a popular film star and suggesting other popular film stars. This is the El Dorado of semantic search, and it is great to a certain extent, but is not without its problems, for example Johnny Holland or Johnny Hallyday won’t get a chance of being suggested because they don’t fit the semantic class of “hollywood film star”. Funneling towards the most popular expressions is still firmly in place.

Spezify for example does something interesting, it displays related search terms next to the initial search term, so when you click on them the corresponding search is executed. I must admit I am puzzled as to how they determine which terms to display, as sometimes the relation is really unclear. Regardless, the result is you can certainly enjoy a chase down the proverbial rabbit-hole by surfing the suggested search terms. A very successful serendipity tactic, albeit overly random at times.

Categorisation – revisited

A few years ago while working on the information architecture for a VOD catalogue of independent cinema I noticed their films didn’t fit the traditional genres (drama, action, comedy,…) so devised a different system based on assigning multiple tags to each film and proposing a navigation system that would let users switch from tag to film to tag. This enabled a high degree of serendipity without the unwanted randomness. This wasn’t implemented but it’s certainly a viable model.

I built a prototype to test the idea and after some experimentation came up with a second model similar to Spezify’s but with a distinct difference: As you search for a keyword, you receive a set of results, each one with its related set of tags. These tags become the suggested tags. Clicking on a suggested tag adds it to the search query, and filters the results. So it’s like progressive filtering, the suggested keywords get added to the mix as your search query grows in size. This model provides a high degree of serendipity without the unwanted side-effect of making suggestions appear random, there is a logical progression as the user creates a growing query. Here is a screenshot of the prototype:

Social recommendation – What else?

Other possible tactics are…

Following strangers

Instead of presenting recommendations based on a user’s peers, show instead recommendations based on the preferences of non-peers, and identify them as such.

Parsing multiple sources of information

During UX Lisbon 2011 Chris Fahey suggested having an app collecting a user’s information from multiple platforms (eg facebook+twitter+last.fm+amazon+netflix) and using the information from multiple sources to make more intelligent suggestions. I suspect Google is about to pull this sort of trick in the near future, if your Google account becomes your identifier across several products (eg. google+, picasa, mail,youtube, maps), it would be easy to combine it all and make more accurate predictions. The prospect will send a chill down the spine of privacy advocates…

Friend of a friend of a friend

Considering the 6 degrees of separation theory, which has some flaws but is nevertheless interesting for our purposes, a system could make recommendations based on indirect relations, for example, showing what films a friend of a friend likes to watch.

Seeing the world through someone else’s eyes

Twitter has recently introduced a nice little feature which lets you see Twitter the way someone else does, and jump between the people you follow. What if this was extended to the people you’ve never heard of? Google+ is doing that really well, you can merrily jump from profile to profile in a highly serendipitious way.

Polysemy

Because images can have multiple meanings and are more open to interpretation than textual information, they can be used as powerful suggestion mechanisms. If combined or sequenced in certain ways they encourage free association of ideas and discovery.

Get our more

It’s good to let go of digital environments and let real life surprise us. Encourage your users to go out more and place themselves in situations they’re not usually comfortable with, nudge them to venture beyond their comfort zones. Location-based mobile apps are the key here.

There’s a lot to be done in this area and I encourage you to take these issues in consideration when you’re next creating your design solutions, and remember how algorithms can limit user’s choices in a detrimental way.