Communication Breakdown

We all depend on gestures to understand words – and those gestures are somewhat unique each time we make them, says a new study.

In the Future, we'll all basically be playing The Sims all the time.

These discoveries could prove highly influential as virtual-world interactions play greater and greater roles in our daily lives, because they help demonstrate not only that gestures form a crucial part of our interactions with one another, but that those gestures must be tailored to the specifics of the moment.

For years, studies have emphasized the importance of gestures in communication – in fact, research on the complex gesture-“language” of chimpanzees strongly suggests that human gestural communication was already rich and complex long before the first true words were spoken.

But now, as the journal PLoS ONE reports, a team led by the Max Planck Institute’s Trevor Dodds has examined what role gestures take when humans communicate through avatars in virtual environments.

The researchers studied the results of cooperative games in which partners could only interact via their avatars. The players had to act out the meanings of words for one another, and scored points for correctly guessing the words their partners were acting out. Sometimes the players controlled their avatars’ movements directly through virtual reality suits; sometimes the avatars didn’t gesture at all, or acted out pre-recorded gestures.

The team found that people succeeded in cooperating best when their own movements directly controlled their avatars’ gestures – and that feedback from the listeners‘ body language was just as important as the gestures of the speaker:

Participants used significantly more hand gestures and successfully described significantly more words when nonverbal communication was available to both participants … [and] participants’ performance was significantly worse when they were talking to an avatar with a prerecorded listening animation, compared with an avatar animated by their partners’ real movements.

In short, pre-recorded gestures don’t seem to cut it when we’re trying to act out an idea for a partner. It’s a little tricky to tell just what the implications of this finding are, though; since a wider range of pre-recorded gestures might improve those results.

But then, the study also found that players used significantly fewer gestures overall to communicate in the virtual world than they used when playing the same game in real life. This hints that, in ordinary real-world conversations, we probably depend on a lot more nonverbal feedback cues than we might realize. As the study puts it:

It is important that nonverbal communication is bidirectional (real nonverbal feedback in addition to nonverbal communication from the describing participant).

Emerging technologies make this research more timely than ever – a growing number of patients who suffer from paralysis are gaining access to thought-controlled hands, and new experiments have demonstrated that – with the right wiring – monkeys can literally feel computerized objects they grasp with a virtual “avatar hand.”

It isn’t just flesh and robotics that are merging – it’s the boundary between the “real” and the “virtual.” It won’t be long before mixed reality is the reality in which most of us live every day. To a certain degree, that’s the reality in which you and I are interacting right now.

Of course, I wasn’t looking to you for any real-time feedback as I wrote this article (some in the comments section wouldn’t hurt, though). But as real-time virtual interactions become more commonplace, those considerations will become increasingly important to the ways in which we understand each other – and understand the ways in which we understand each other.

Pardon the awful pun, but it’s virtually guaranteed.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s