Why gesture is the next big thing in design


It happens more often than I would have anticipated. People talking to their phone, mid-conversation in order to let Siri figure out an argument, book tikcets or convert currencies. To this day I have refused to talk to an inanimate object such as a smartphone as if it where my personal valet. Hands down (yes, pun intended), NO to sonic-interaction.

The kind people at IDEO seem to agree with me and wrote this uplifting piece on more gesture based interaction. As you might have guessed it opens up a new door of interacting for designers to explore. To me already more relatable then awkwardly babbling in you phone.

Since Siri was introduced in 2010, the world has been increasingly enamored with voice interfaces. When we need to adjust the thermostat, we ask Alexa. If we want to put on a movie, we ask the remote to search for it. According to some estimates, some 33 million voice-enabled devices reported for duty in American homes by the end of 2017.

Thankfully, though, talking into midair is no longer our only—or best—option.

But there are limitations to voice-enabled interactions. They’re slow, embarrassing when other humans are around, and require awkward trigger phrases like 'Okay, Google' or 'Hey, Siri'.

The new iPhone introduced a camera that can perceive three dimensions and record a depth for every pixel, and home devices like the Nest IQ and Amazon's Echo Look now have cameras of their own. Combined with neural nets that learn and improve with more training data, these new cameras create a point cloud or depth map of the people in a scene, how they are posing, and how they are moving. The nets can be trained to recognize specific people, classify their activities, and respond to gestures from afar. Together, neural nets and better cameras open up an entirely new space for gestural design and gesture-based interaction models.