Gestural user interfaces allow people to interact with products and systems using familiar everyday physical movements. While tapping a touch screen is a familiar interaction, gestural user interfaces do not require physical contact between the user and the product. For example, relatively inexpensive cameras and optical sensors can detect proximity, direction and degree of movement that can be used to drive interaction. Moreover, gestural user interfaces allow product designers to create very usable and easy to learn interfaces by mapping culturally recognizable gestures.
For example, Bresslergroup recently completed the design and engineering for the launch of the CLEANCut Touchless Paper Towel Dispenser. This product features a gestural user interface which allows consumers to use a combination of gestures to select the size and then cut to desired length a paper towel, with any type of conventional paper towel roll. Infrared sensors in two zones detect when to start and stop the towel dispensing, and then when to cut it:
Although gestural user interfaces use familiar physical behaviors, like hand motions, as the basic building blocks of interface design, this does not mean that such interfaces are limited to simple tasks. More complex interactions can be accomplished via combinations of gestures and the use of both familiar and learned gestures. As a consequence, designing usable, effective gestural user interfaces requires a clear understanding of both interaction design and human physical capabilities. It also requires a synthesis of these two fields in new ways. For example, Rob Tannen recently presented guidelines on Ergonomics for Interaction Design, in the video presentation below:
As the number and variety of gestural user interface applications has exploded, design researchers have been focusing more on the physical aspects of human-artifact interaction. The potential benefits of gestural user interface - simplicity, ease of learning, memorability - are valuable to end-users, but also provide an emerging opportunity and challenges for the researchers who study them.
This presents great opportunity but also risk. Most interaction designers lack a sufficient understanding of the ergonomics and kinesthetics of the human body. Moreover, there is a chasm in our understanding of how physical actions relate to information-based tasks. We learn complex physical interactions for physical activities: riding a bicycle, painting a picture, even cooking. But our physical interactions for information-based tasks tend to be relatively basic: turning a page, scrolling a wheel or pressing a button.
There are a handful of cases that apply physical dexterity for information-based tasks, but they are few and far between: using an abacus, telegraph, or more recently, texting & typing. But now, there are emerging solutions for leveraging the capabilities of the body to more effectively interact with information.
Make the interactions too basic, and we've just replaced a mouse with a finger--convenient perhaps, but not necessarily an improvement. On the other hand, make the gestural user interface too complex or demanding and run the risk of excluding people who don't have the capabilities or proficiency to perform like a rock star.
Creating effective physical-to-digital interactions for a good gestural user interface is a unique challenge. With the great human diversity in physical capabilities due to age, gender, physical condition and motivation, there is rarely a "one size fits all" solution for gestural user interfaces. A human-centered approach involving the collaboration of interface & industrial designers with experts in human factors & ergonomics is necessary. Ultimately, with the continued spread of physical interaction technologies and metaphors, it is essential for designers to not only "know thy user," but to know their users' bodies.
Typically, design research documentation is a cumbersome process of rapid note-taking and rigorous reviews of video recordings. Over the years we have identified and developed tools and technologies that can improve the efficiency of design research data collection. And we see the next wave of user research tools not only focusing on gestural user interface, but actually enabling the design researchers to document behavioral observations through their own physical interactions.
Case in point - at Bresslergroup we have been trialing the Nintendo Wiimote controller in conjunction with Techsmith's Morae software as a wireless data tagging system. Wireless data tagging is an activity that allows one or more researchers to mark key observations, synchronized in real-time. In fact, wireless tagging was a key element of our FieldCREW concept research tablet.
Gestural user interface is part of our proof-of-concept for FieldCREW. To optimize the system, we have configured the various buttons and controllers to correspond to task-specific behaviors. For example, the directional controller can correspond to the direction of an observed person's physical motions, while the other buttons can be mapped to specific tasks or activities. As a result, a skilled "tagger" can unobtrusively document events in real-time with a single-hand and without interrupting their line of sight to the activity (versus having to look down to write notes). All of the tagged events become markers corresponding to a synchronized video-recording in Morae.
This is more than a gain of efficiency or convenience. It is an enhanced means to observe and record physical behaviors, as it changes the role of the observer from a somewhat passive information gatherer to an active, real-time participant, even giving game-like qualities to the research tasks (e.g. when the user moves left, I push left).
