Friends of the Nonverbal Communication Blog, this week we present the paper “Racial Identity-Aware Facial Expression Recognition Using Deep Convolutional Neural Networks”, by Sohail, M.; Ali, G.; Rashid, J.; Ahmad, I.; Almotiri, S. H.; AlGhamdi, M. A.; Nagra, A. A. and Masood, K. (2021), in which authors investigate the possibility of including, thanks to a software, the ability to consider ethnic groups when it comes to analyzing facial expressions.
Because manual analysis of facial expression is sometimes kind of time-consuming, different software is being used more and more to automate the process. However, authors wonder, do they consider the differences between ethnic groups and their faces?
The objective of a facial expression recognition system is to recognize the emotions that they show, because there are a series of muscular movements of the face that are associated with certain emotions and can make us infer what a person feels.
These emotions are usually happiness, sadness, surprise, anger, fear and disgust, although contempt or neutral expression are sometimes included.
Although many studies have focused on facial expression recognition using static images, this can be very difficult for three reasons, as authors explain.
First, because the variations in facial structure between subjects from different cultures make the classification task pretty complex in some cases.
Second, because the similarity between expressions may be significant, and therefore a challenge to recognize each one accurately.
Lastly, different subjects may present variations in the expression of their emotions due to their facial appearance and their biometric forms.
In general, authors consider that the variability of facial structure between cultures could lead to incorrect recognition of facial expression because the image of an emotion in one culture may be different from the image of the same emotion in another culture. For example, it is known that members of different cultures can express levels of arousal to an emotion that won’t appear in another culture.
Therefore, authors believe that including a description of racial identity in automated facial recognition software models would make the process more reliable.
Specifically, authors develop in this work a new deep learning technique: the Racial Identity Aware Network (RIA-Net) learns facial expressions from images and extracts racial identity features from a previously trained racial identity network (RI-Net). -Net). The latter is trained using multicultural data from Japanese, Taiwanese, American, Caucasian, Moroccan people…
In addition, they use a model based on convolution neural networks, which has previously been successfully used by experts such as Pons and Masip, for facial expression recognition.
Authors think that it is very important to consider ethnic. It has been shown that the representation of facial expression is not only influenced by muscular deformation of the facial structure, but also by many other social factors such as culture, geography, or ethnic group.
How was the study carried out? Well, a recognition system for facial expressions associated with seven emotions was proposed. It knew sadness, happiness, anger, fear, surprise, disgust and neutral emotion.
Then, five different cultures were considered: Moroccan, Caucasian, Taiwanese, American and Japanese. To do this, images from country-specific databases with faces of native people were extracted.
The findings show, in the first place, that the highest percentage of misunderstanding arises among the emotions of anger, sadness and fear, the opposite occurring with happiness and surprise, where there were 100% correct answers.
The proposed method achieved an accuracy of 97%. Under the same conditions, without using racial identity traits, the accuracy dropped to 93.28%. These results show that the use of racial identity traits in the recognition of facial expressions significantly improves the results.
The current pandemic situation has made online communication much more common. In addition, globalization has facilitated communication between people from different parts of the world and, therefore, different cultures. Facial expressions play a very important role in this regard, so it is especially important to pay attention to multiculturalism when it comes to identifying emotions through the face.
If you want to know more about nonverbal behavior and how it influences our personal relationships, visit our Nonverbal Communication Certificate, a 100% online program certificated by the Heritage University (Washington) with special discounts for readers of the Nonverbal Communication Blog.