Physical Interaction Design

From here we move to relatively simple, ubiquitous traditional physical interaction design controllers - buttons, keyboards, knobs, switches, levers - the stuff of mechanical and electro-mechanical devices that industrial and gui designers have been working with for years. These controllers are typically binary (on/off), or at least incremental (having multiple, discrete states). Most existing touch screen gui, such as bank ATMs, would fall under this category.

We then go from discrete, to continuous controllers, enabling multiple actions and greater flexibility. The computer mouse was a breakthrough for human-computer interaction in this context as it supports various types of physical interaction and interfacing from a single control device. In fact, while keys and buttons are typically designed with a specific function in mind, the mouse provided the opportunity for new user interfaces to be created for defining its functions. Gestural interfaces, from multi-touch gui screens to the Wii are also examples of this flexible, "open" physical interaction category. These are the "new " physical interaction devices that are opening up new possibilities for interaction designers.

We might imagine a Minority Report based interface as the ultimate extreme at the far end of the physical interaction spectrum, but as pictured in the video above, it is only limited to gestural hand movements. What about more complex bodily interactions combining other limbs, postural movement and line-of-sight? This largely unexplored territory might be best understood by observing how we use our bodies in the most dynamic and complex ways. Musicians, athletes and dancers may be a more valuable source for developing future physical interaction ideas than science fiction.

Note that the continuum described above, while not by any means arbitrary, was not based on a well-defined metric that quantified greater or less physical complexity. If we were to do so, degrees of freedom would be an appropriate place to start. A degree of freedom can be defined as any independent direction in which movement is possible. A human finger has four degrees of freedom, made up of the extension/flexion of the three joints, as well as side-to-side movement. Combining the individual degrees of freedom of the four fingers, thumb and wrist gives the hand 26 degrees of freedom.

Hypothetically, we could apply this to the entire human body to specify the maximum level of complexity for any single physical interaction, or sequence of interactions. The total degrees of freedom for a fully functioning human is 1380. In theory, we could go back to any physical interaction and quantify the (minimum) amount of movement required to come up with relative complexity measures. But it actually gets more challenging as complexity is more than just the sum of the degrees of freedom, and would depend on the particular combination of movements, etc. In other words, it's an interesting idea, but requires a lot more thought to pursue practically.

Another important consideration is the relationship between physical interaction design inputs and the associated outputs in a user interface system. Current discussion of gestural interfaces is primarily focused on using physical interaction to control virtual objects - a way to make the digital world more tangible. But physical interaction interfaces can also be used to control physical systems, and not just in the literal sense.

For example, Intuitive Surgical's da Vinci surgical systems represent the leading edge of commercialized physical interaction devices. As depicted in the video, the systems "translate and filter" a surgeon's precision hand motions into physical motions of surgical robot manipulators. This requires a two-way physical interaction where the user not only provides physical output, but receives haptic input such as resistance to force. So it's actually a physical-to virtual-to physical loop.

There are threads towards developing a taxonomy of physical user interaction types:

There is the complexity of the physical movement, characterized by the number and type of degrees of freedom involved.
The output of the physical interaction, resulting in either virtual actions, physical actions or a combination.
The directionality of the interaction: either one way from user to system, or bi-directional between user and interface