Siftables are small devices which have a graphical display, a number of sensors and wireless communication capabilities. They are small tangible user interfaces which can function individually or in a group, and can be manipulated to interact with digital information and media.

Siftables is a project from David Merrill, a fourth year Ph.D. student in the Fluid Interfaces group at the MIT Media lab, and Jeevan Kalanithi, a designer and technologist from Taco Lab. For the design of Siftables they were inspired by how humans use both hands and all fingers efficiently when manipulating large quantities of small objects. This is a capability which isn’t utilized in today’s human computer interaction. Siftables however does make use of it, and it does it in an extraordinary manner.

Here is a video in which Siftables are demonstrated:

David Merrill’s presentation at TED about Siftables is also definitely worth watching.

Siftables totally makes use of the five themes for interaction design described in the research paper from the Standford Univeristy HCI Research group which we wrote a post about in december. It’s exciting to see how these human capabilities are finding it’s way into human-computer interaction. Are we going to find this in our living-rooms 15 years from now?

We always try to apply user centered design, thinking what the world should be like for people. But what if we would turn it around. What does a person look like to a computer? That’s the question Dan O’Sullivan and Tom Igoe asked themselves.

On the left you see the answer to Dan O’Sullivan and Tom Igoe’s question (from the book “Physical Computing”) of “What does a person look like to a computer?” In other words, to most computers with a mouse, keyboard, and graphical user interface “We might look like a hand with one finger, one eye, and two ears.” (2002, ppxi-xix).

After the 40th anniversary of the “Mother of All Demos” (where Doug Engelbart premiered that very interface), it’s worth taking a look at a few newer devices and how they might ‘see us. Let’s take a look…

Remote controls

To a remote (and its related TV/VCR/dimmer lights), all that you are is a finger. But, like a person with a voodoo doll, that finger through the remote wields a great deal of power – able to put something to sleep with a single, well placed prod. (Try thinking of that the next time you’re cursing at your DVD player.)

Mobile phones

Early mobile phones (portable phones) saw us as a mouth, ear and finger – usually from a car where they were installed. Once their batteries were small enough to be truly mobile, they would suddenly have become aware of us having arms (and pockets), and with the advent of the screen, us having eyes.  However, the most “eye opening” moment for mobile phones has to have been with the advent of flip phones (as  popularised in The Matrix). At that point, mobile phones could see that we had hands capable of manipulating them… something the iPhone has lost.

iPhone

And on to that darling of multi-touch, the iPhone. Ironically, in some ways, the iPhone sees us less than many mobile phones – we might have quite a few tentacles (after all, it’s all about stroking, not clicking), but no opposing thumbs. That said, we’re definitely something big and powerful, since we can affect the accelerometers by rotating the device. And add-ons like the Heart Monitor Application might make an iPhone realise we are alive ….

Wii

Finally, the Wii is interesting in that it begins to not only see us as entire bodies (fingers, arms, eyes), but even as more than one person.

So why might we want to look at devices in this way? According to O’Sullivan and Igoe, “…to make the computer a medium for expression, you need to describe the conversation you what to have with (or better yet, through) the computer” (pix). More pragmatically, it’s a quick way to challenge your own perceptions about how a device might work. After all, it’s easy to forget that not everything sees the world the same way as we do.

Photos: litmuse (HAL), ThunderChild tm (remote), debagel (Nokia NHK-6AX), William Hook (iPhone), chispita_666 (Wii). All under a Creative Commons Licence

The big advantage of designing on a computer is the ease of changing things, like color and shape. But, especially for 3D objects, it has some disadvantages. You don’t really get a feel for the object: What does it look like when I walk around it? How does it feel when I hold it in my hands? With Claytronics technology this problem could be solved.

Claytronics technology is about reconfigurable nanoscale robots. All these tiny little robots (catoms) are part of a much bigger apparatus. The Carnegie Mellon University together with Intel are currently researching this technology. Though it might seem somewhat futuristic, they are confident that it can be realized and they’ve got Moore’s Law to back it up.

Here is a concept video to get a better understanding of where this is actually going. The video is made by students from the Carnegie Mellon University.

Now as I said, it seems somewhat futuristic, but these nano-based technologies aren’t that far away anymore as most of us might think. Earlier this year I attended an absolutely jaw-dropping presentation on nanotechnology by prof. dr. ing. Dave Blank from the University of Twente (the Netherlands). He’s a full professor Inorganic Materials Science of the Faculty Science and Technology and the Scientific Director of the MESA+ Institute for Nanotechnology. In his presentation he showed to what extend this technology already has been developed. We were amazed to see that a lot of it is ready to use. They’re currently in the final stages of checking if it’s save, and then it’s ready to go.

I find it to be an exciting development as well as a scary one. Technologies like Claytronics offer so much interesting possibilities. It results in a new way of human-computer interaction which is so incredibly organic that you almost forget it still is human-computer interaction. A new marriage between the tangible and the intangible. Imagine this technology in your phone for example. As a matter of fact, Nokia allready did that with their Morph concept. It’s nothing less than revolutionary.

What I find to be a bit scary about it, is that it consists out of (programmable) particles which can’t be seen with the naked eye. Particles that can be inhaled easily or get into the water supplies. 15 to 20 years from now might be the very first time in history that we can get (physically) sick from a computer virus.

One of the best reads on tangible interactions I came across lately is a paper from Standford Univeristy HCI Research group. It is titled: How Bodies Matter: Five Themes for Interaction design. In it they use theories from psychology, sociology and philosophy to come up with five themes for designing tangible interactions. Because they bring all these sciences together on one subject, it offers some interesting insights in the field of tangible interactions.

The themes

1. Thinking through doing - Here they discuss how the body and mind are deeply integrated and how they co-produce learning and reasoning. It makes you realize how important tangible interactions are throughout our lives. Thinking/learning through doing is the way we learned how to walk, talk and ride a bicycle. Even the way we’ve been able to create a general understanding of our direct environment is a product of such interactions.

Gestures also play a big role in this process. It is said that next to just being a visual or sensory aid, they also lighten the cognitive load. Even blind people use gestures. A interesting conclusion they draw here is that systems which constrain our gestural abilities are likely to hinder our thinking and communications. They make a reference here to the mouse and keyboard.

Another interesting point they make is about epistemic action (the way we manipulate the environment to get a better understanding of things). In a game of Tetris, when someone gets better, the amount that he/she rotates the block before it is placed between the others, actually increases instead of decreases. It’s the same with scrabble; players that are better, move around the letters on their plate way more often than others.

2. Perfomance – With this they mean the complex performance of what our bodies are capable of. How we are able to master the use of an object to such an extend that it becomes an extension of ourselves is quite extraordinary when you think about it. Surgeons and musicians are an example they use. They do this every day, using their hands to operate their tools. Yet such rich interactions are hardly used in today’s human-computer interaction.

3. Visibility – This is about the the role of objects in collaboration and cooperation. Here they refer to a study about the use of flight strips in air traffic control. Air traffic controllers used these paper flight strips to make a physical representation of the airplanes. The outcome was that the person was managing the strips he had laying down in front of him thereby managing the air traffic, not the other way around. Visibility facilitates coordination.

But there are more ways visibility plays a role. Think about how we can learn skills by looking at someone else performing a task. Or why we still go to concerts while we can also listen to the music at home.

4. Risk – Tangible interactions come with a certain risk. Where we can undo our actions in a digital environment, in a physical environment they are often permanent. Therefore it demands more commitment and trust to perform a physical action compared to a virtual one.

Because there is more risk involved, people also tend to have a bigger sense of personal responsibility. That’s partially due because the consequences of their acts are visible and there for others to see. An interesting example of this which isn’t mentioned in the paper is Hans Monderman’s approach to road engineering. He was a Dutch road engineer and innovator who came up with the idea to make roads look more dangerous, and as a result to that, they became more safe. This was al due to the fact that people payed better attention.

5. Thickness of practice – This is about the tangible vs. the intangible. Technology has provided us with reliable and accurate systems. A intangible replacement for the tangible. The point they’re trying to make here is that though intangible solutions have their advantages, we should take great care before replacing the tangible with the intangible without reflecting upon it. There is just so much in the physical world we could benefit from.

Where did all the tangible interactions go?

That’s what this paper made wonder. How can it be that today’s most widely used human-computer interaction is still based on a mouse and keyboard. Tangible interactions play such an important role throughout our lives. Even in our design process, almost every time when we need to get creative or get more insight into something, we result in tangible methods. We start sketching on paper or brainstorming on a whiteboard or with post-its, we create physical prototypes or do field research.

I believe that a tight cooperation of the tangible and intangible can result in a much richer experience. Good progress is being made nowadays. Tangible interactions are starting to enter the field of mainstream human-computer interaction (look at the iphone or the nintendo Wii). However it’s still far from reaching it’s full potential.

The paper is called: ‘How Bodies Matter: Five Themes for Interaction Design’ and is written by Scott R. Klemmer, Björn Hartman and Leila Takayama. It is published in 2006 and can be found here amongst other Stanford HCI publications. It’s an absolute must read for everyone involved in experience design.