Interaction Design

So we have largely been designing two, co-existing, but separate types of products: those that utilize the complex mechanics of the body to transform matter, and those that accommodate capabilities of the mind for processing information. Products that overlap these two worlds, that is, which take advantage of complex physical interaction design to drive information processing, are few and far between.  The abacus the telegraph, and perhaps texting, comes to mind as representatives of physical interaction design. Musical instruments play an interesting role in this context where technical sophisticated body control is applied to the creation of sound, but even this is for artistic or entertainment purposes, rather than concerning the analysis of information in the scientific or business sense.

But now, we are starting to see the emergence of products that hint at using the mechanics of the body to interact with complex information. For example Jeff Han’s multi-touch, interactive data wall, and virtual reality simulations that allow scientists to “feel” the forces between molecules. But even these interaction design interfaces are only scratching the surface of what is possible from a physical interaction point-of-view. Getting to this point has taken a long time because it is challenging to track and quantify the multiple degrees of freedom of movement of the body. But it is also the result of our divided design processes, where physical interaction design and cognitive interaction design have traditionally been designed by different people with different expertise, at different times – the ID team, the gui design team, etc.

Moreover, such divided interaction design processes are themselves the result of 20th century psychology that treats humans as information processing machines. As a consequence we think of human activity as comprised of discrete, sequential steps of thinking and then acting – I see something, then I reach for it. In other words, a built-in division between cognitive interaction and physical interaction. As a result it’s easy to see why we divide the world the way we do.

But there are alternative perspectives on human behavior, in particular the ecological psychology of J.J. Gibson. Gibson coined the term “affordances” which is (mis) used and abused by gui designers today. But affordances, the relationships between people (or other organisms) and artifacts, are just a part of a larger “perception-action” framework. In this view, perceiving or sensing information is a physical behavior itself, not just a means to drive a subsequent physical action. Likewise, physical behavior drives perception – the two are connected, not divided, resulting in a perception-action loop.

The implications of this for physical interaction design are subtle, but important. Taking into account the physical interactions someone takes to acquire information is useful for understanding and determining what and how to display information. For example, rather than designing products where all information is accessed from a single point, information may be distributed across locations, where the location itself is informative above and beyond the content. We see this emerging with augmented reality applications, where the user’s particular activity, such as walking to a place in a city, provides location-specific information. The format and content may be driven by variables such as the person’s posture and gait (e.g. in a hurry or browsing), direction of approach, and of course their physical characteristics such as eye height and reach.

To get to the point where we can design gui and systems to take advantage of complex physical  interactions will require taking a new look at how the fields of kinesthetics, anthropometrics and optical flow, relate to interaction / gui design. These fields will be as important to designers as information architecture and form giving are today. As a starting point, I've been exploring the role of ergonomics in contemporary gui design, to better relate the divided fields of physical and cognitive product design.

A continuum of physical interaction design would range at one end from non-movement to that which requires significant, complex body movement. At the first extreme we could include theoretical telepathic control, where no visible physical interaction is evident. As we move away from that end of the spectrum we have existing and emerging neuroergonomic interfaces that rely on measurement of electrical potentials, cerebral blood flow, and MRI imagery as triggers for action. 

The next step towards physical interaction design are devices that measure or track relatively small motor movements and translate them into interface actions. For example, Emotiv System's forthcoming Epo device can translate facial muscle movement into expressions for online avatars. Eye tracking systems, already commonplace in supporting the physically disabled, track eye movements in place of mouse/keyboard controllers.

Continue >>
Bresslergroup directly >>