Part 2: What’s Next in Interaction Design — Dispatch from Interaction 16

(Read Part 1: What’s Next in Interaction Design — Dispatch from Interaction 16.)

Today is the last day of Interaction 16 in Helsinki. Deciding which talk to attend when there are three parallel tracks feels like playing roulette.

Will that mundane-sounding talk turn out to be unexpectedly intriguing? Will the one that sounds great be disappointingly trite?

interaction 16

Magical UX

This was the nature of my concern walking into Josh Clark’s talk, “Magical UX and the Internet of Things,” this afternoon. Three days into an IxD conference, buzzword weariness begins to set in.

The moniker “Internet of Things” has never been my preferred way to describe the newfound ease with which we can infuse sensing, connection, and computational ability into everyday devices and environments to improve functionality and experience — but given the mouthful that is, we’ll stick with IoT.

Turns out Clark of Big Medium gave a talk that was just what I needed. He started with an example of combining voice and gesture in an interface:

interaction 16

There’s been much discussion so far about agency — who is in control? Software or the human? How do we transfer control from one to the other? How can we achieve desired outcomes without being burdened with decisions and interface? Can we trust systems to act on our behalf?

By making the comparison to magic, he hit upon a sweet spot. Magical UX keeps us in control as an instantaneous extension of our will.

It gives us a superpower. It imbues ordinary objects with magical qualities. We don’t discard our old mental models for how things work, but we enhance them.

He used the example (from the world of magic) of Hermione’s magic bag — it’s still a bag, it still holds stuff. The magic is that it holds way more than it should. An example from the real world: the Niemann Marcus Memory Mirror  (previously only available to Snow White’s evil queen).

interaction 16

This Samsung induction oven uses LEDs to simulate a flame, tapping into the existing mental model for cooking.

The takeaways: Design things for their essential “thingness.” Ask yourself, “What if it was magic?” The goal of the interface has always been to create new abstractions that simplify the mental model of the system being used.

It’s not a challenge of technology, but one of imagination. And we’ve already got a pattern library for magical interaction models: we’ve been imagining magical experiences throughout history. Now it’s time to bring them to life.

More Magical Links

• The Wand Company
• Grab Magic
• A Sandtable as Fun as a Sandbox at the Army Research Lab
• Happy Together by Larry Legend and Josh Clark
• Frog Design’s Room-E
• Wayfindr by ustwo

Designing Human

By now we all know the importance of designing with empathy. But designing for empathy is another theme being discussed across various topics this year.

interaction 16

Kate Darling of MIT Media Lab dug into our innate tendency to anthropomorphize while talking about “Robot Ethics and the Future of Human-Robot Interaction.” We project intent into the behaviors we think we observe, as noted in the 1944 Heider and Simmel study  (Heider, F., & Simmel, M., 1944, An experimental study in apparent behavior. The American Journal of Psychology, 57, 243-259).

Robots are designed to exploit this tendency. It can be used for good, as in Nau, a therapeutic robot who helps autistic children and their parents bridge communication gaps; and Parro, a robotic seal who assists the elderly. Of course there are also opportunities for the not so good. These characteristics can be used to exploit our emotional response. As she put it: “What if your sex robot had in-app purchases?

interaction 16

Along similar lines, Jonathan Foster, who works on Microsoft’s Cortana team, talked about what it means to create a true conversational interface, complete with personality. They had to create a character for Cortana — a personality. What Cortana says, how she says it, and more fundamentally, who she is. To do this, he works with a team of playwrights, novelists, script-writers, and poets — not your typical interface design team.

interaction 16

To build trust and create an emotional connection with the user, her personality needs to be clear. Ambiguous personalities, Foster pointed out, are universally disliked. People will assign a personality to a system that communicates with them (back to anthropomorphism again). The Cortana team had to create a robust character and set of responses to all sorts of questions.

Ask her what her favorite TV show is, and she’ll tell you it’s “Star Trek: the Next Generation.” Other topics are trickier: Tell Cortana, “I’m gay.” Her response? “Cool. I’m AI.”

quartz app - interaction 16

An interesting twist on the traditional conversational interface was pointed out by Leando Agro, from Design Group Italia. The Quartz app delivers news to (mostly younger) users, by framing it like a back and forth texting-style conversation.

More Cool Robot Links

• Jibo, a social robot for the home
• Ulo owl surveillance camera, by Vivien Muller

Passenger or Driver?

John Rousseau of Artefact discussed the challenges surrounding the introduction of autonomous vehicles. It’s not an issue of technology, it’s a problem of people. Again, the idea of agency — who is in control? — is the crux of the issue. There are basically two camps:

google moonshot - interaction 16

1. Google: Humans are not a reliable backup for automated driving systems. Fully automated is the way to go. (I agree — see my post, ‘Can Interaction Design Save the Internet of Things?‘)

2. Auto industry / everybody else: We should slowly and gradually introduce partially autonomous features but give drivers ultimate control — at least for a while.

driverless 1 - interaction 16

The transition of agency from software to driver is the most dangerous moment and a tricky UX problem. Rousseau walked us through a few concepts that he and his team had come up with to handle this transition.

Designing for Perspective

Tricia Wang from Constellate Data discussed the pervasiveness of singular perspective, particularly in western culture, and how it colors everything it comes into contact with.

Big Data, which claims to be raw and unbiased, really isn’t when you look under the hood. It’s a representation of reality, but reflects a singular perspective. Like a mirror (which reflects same), it can reverse, distort. Deciding what to measure, how to measure, and what to make of the data — all involve a perspective. And singular perspectives simply cannot represent unbiased truth.

perspective - interaction 16

She pointed out the difference between Big Data, which is quantitative, and Thick Data, which is qualitative — based on ethnographic-type research. Thick Data, she says, can get closer to the truth as it seeks to understand various perspectives.

As we move from explicit design, which is based on an individual designing from his own perspective, to generative design, which is more goal-based, the role of the designer shifts to something more akin to an orchestra conductor. The designers become ethnographers, gathering the inputs of various perspectives to feed into the solution.

Signing Off for Now

There’s so much I haven’t been able to cover, but there’s a closing party to attend and long flights ahead. But there’s a lot to think about. Are we making magic? Are we preparing the world for AI by simulating human interfaces? Who is in control?

The next time I get a chance, I’ll ask Cortana.

Worst Interaction So Far:

Every time I meet someone, I have to explain that yes, I’m from the country where Donald Trump could actually become president. And yes, I’m from the city that killed HitchBOT.

(ICYMI, check out my first dispatch from Interaction 16, Part 1: What’s Next in Interaction Design.)