Friends of the Nonverbal Communication Club, this week we present the paper “Nonverbal Communication in Virtual Reality: Nodding as a social signal in virtual interactions”, by Aburumman, N.; Gillies, M.; Ward, J. A. and Hamilton, A. F. C. (2022), in which authors carry out a series of experiments to know how the nodding affects the perception that users have of human avatars in virtual reality contexts.
Face-to-face interaction is a central part of human life, used to convey ideas, share information, understand others’ intentions and emotions, build trust, make decisions….
An important goal for computational science researchers is the design of virtual environments, including virtual humans and immersive virtual reality contexts, that can simulate a real face-to-face conversation. It is also an important goal for researchers in psychology to understand how humans behave during interactions and to test theories about which aspects of these interactions are most meaningful.
Whether in a physical or virtual setting, human communication involves both verbal exchanges and nonverbal behaviors.
Nonverbal communication is an effective and expressive tool used to send and receive social signals that humans have been using for thousands of years before the ability to communicate with words was developed. Therefore, both the analysis and synthesis of nonverbal communication is an essential part of human-computer interaction research.
Although physical communication is still more powerful, modern communication is often mediated by technology, and it takes place virtually.
Virtual reality is a digital form of communication that can facilitate the creation of immersive real-time interaction and enhance social presence in virtual environments.
In the present study, virtual reality was employed in the experiments as the authors felt that it had unparalleled potential to impact the future of numerous sectors, such as virtual conferencing, education, consulting, social rehabilitation, medical care….
They also included nonverbal communication, which refers to such disparate aspects as nodding the head, maintaining eye contact, leaning forward or backward, body orientation, among many others. In particular, nodding plays an important role in regulating an interaction, signaling who should take the floor, for example, or whether or not someone is interested in a particular item.
This type of signaling is commonly referred to as backchannelling, and often occurs to send subtle messages in a face-to-face interaction. Including this element in virtual environments, therefore, can be very important to make the interlocutor feel comfortable and heard.
In this paper, authors implement several experiments involving virtual interaction between a human-controlled avatar and a virtual human whose behavior is controlled by a computer program. In these experiments, authors focus on four different types of nonverbal cues that are very important in human face-to-face interaction: blinking, head nodding, facial expressions, and gaze shifting. In addition, they specifically manipulated the nodding behavior between two different virtual humans.
The experiments were conducted at the social interaction laboratory at University College London. Data could be collected from 21 participants, of which 15 were female and 6 were male, with an average age of 24 years.
The style of the virtual avatars was unrealistic, cartoon-like, as this type of virtual human is preferred over more realistic ones.
In the first task, participants were told that they would have a conversation with two different virtual humans in virtual reality, and discuss a series of facts about some U.S. states. The participant meets the first virtual human (Anna). She introduced herself, and asked the participant to introduce him/herself. Then, Anna performed a 45-55 second monologue, where she read facts about a US state and then, for 35-45 seconds, Anna and the participant discussed. After that, the process was done in reverse. In total, the participant had to complete four attempts with Anna and four with the other virtual human, Beth.
Authors designed these two virtual humans to provide identical blinks, facial expressions, and changing gaze behaviors. The only difference between the behavior of the two virtual humans is that one of them manifested a naturalistic nodding behavior that depended on the actions of its partner, while the other only exhibited a preconfigured head movement.
The second task used a virtual maze to implicitly measure the participant’s proximity, trust, and attraction to the virtual humans.
Virtual humans Anna and Beth were placed at decision points in the maze; and the participant could choose to approach one or the other for advice on how to complete the activity.
A positive impact of naturalistic nodding was found, showing that participants liked more, and trusted more, the virtual human who nodded in this way, as she was rated significantly higher than the other virtual human.
When participants were asked what virtual human had shown more attention to what he/she was saying, opinions continued along these lines, and the virtual human with a naturalistic nod was perceived as more engaged in the interaction.
Furthermore, in the maze experiment, participants were closer to the virtual human who nodded more.
These results support the claim that mimicry functions as a kind of social glue, and that by copying another person’s actions it is possible to generate trust and sympathy.
Future studies could test how this extends to other types of conversation and other social groups, for example, by introducing the variable of gender.
If you want to know more about nonverbal behavior and how it affects personal relationships, visit our Master of Science in Nonverbal and Deceptive Behavior, which you can take in English or Spanish, with special grants for readers of the Nonverbal Communication Blog.
