Tag Archives: facial expression

Human facial expressions may be universal across cultures, AI study found

Credit: Pixabay.

There are more than 7,000 languages spoken in the world today, but sometimes facial expressions communicate much more than words, regardless of your mother tongue. Whether facial expressions and the emotions that underlie them are preserved across cultures has been a subject of great debate for years.

Studies that have attempted to document the universality of facial expressions have typically relied on experiments in which participants had to label photos of posed expressions. If there’s a consensus among participants from different cultural backgrounds about what behaviors reflect “joy”, “anger” or some other affective state, then this would be evidence of universal patterns of behavior.

However, these setups are limited and biased by language, as well as cultural norms and values.

Scientists at the University of California Berkeley and Google Research took a more objective approach, looking to assess human facial expressions and social situations in a more natural context. The team led by Alan Cowen, an emotion scientist at the University of California, Berkeley and a Visiting Faculty Researcher at Google, trained a deep neural network to evaluate whether different social contexts were associated with specific facial expressions across different cultures.

The deep neural network, which is a type of machine learning that attempts to draw similar conclusions as humans would by continually analyzing data with a given logical structure, was initially trained with data from English-speaking people from India, who tagged 16 patterns of facial movement associated with distinct English-language emotions.

Credit: Nature, Cowen et al.

The algorithm was then fed 6 million YouTube videos from 144 countries — a staggering trove of data that is much larger and diverse than the sample sizes of any similar studies before it. The assessment showed that similar expressions occurred in similar contexts around the world.

“There’s a lot of debate about whether facial expressions mean the same things in different cultures. For decades scientists have relied on survey data to address this question. But we don’t know how accurate those surveys are, because there are language differences, because it depends on how you ask the question, and because facial expressions can have multiple meanings. What we haven’t been able to do, until now, is look at how facial expressions are used in the real world. This is the first worldwide analysis of how facial expressions are used in everyday life, and we found that universal human emotional expressions are a lot richer and more complex than many scientists have assumed based on survey data,” Cowen told ZME Science.

The study, which was published in the journal Nature, took four years of hard work to complete and required the development of novel machine learning tools. “When we first got our algorithm working, that was a big moment for me,” Cowen recounts.

In their study, the authors mention how expressions such as “awe”, “contentment”, and “triumph” were associated with wedding and sporting events irrespective of the country where they took place. But each type of expression had distinct associations with specific contexts that were 70% preserved, on average, across 12 global regions.

“The expression of triumph during sports emerges as one of the most universal across cultures, at least in terms of what’s captured in videos online,” Cowen said.

“We found that there was some degree of universality for all 16 of the facial expressions we analyzed,” he added.

These findings point to a universality of facial expressions, which may be biologically hardwired. However, this isn’t the final say on the matter. Since all the cultures included in the study have access to the internet, it is possible that the findings reflect culturally-transmitted facial expressions through globalization.

Nevertheless, the researchers also have strong evidence that facial expressions are mainly driven by biology.

“We recently even found evidence that context-expression associations depicted in ancient Mayan sculptures reflect Western intuitions about emotion, and those predate cultural contact with Western Europe,” Cowen said.

The researchers also found that context-expression associations in Indonesia and the Philippines were closest to the world average, rather than in the U.S. or Western Europe as one would assume.

“We do see some evidence for more dispersed geographic diffusion of facial expressions, and it seems to suggest that they’re a little bit cultural. But overall, a strong case is emerging that some facial expressions are biologically prepared,” Cowen said.

Mice have different facial expression depending on how they feel — a doorway to the origin of emotions

The facial expressions of mice reflect their internal emotional state, similarly to humans. The findings offer a possible neural mechanism of emotions. Credit: MPI of Neurobiology / Kuhl.

There are over 7,000 languages spoken in the world today. And while there is a lot of common ground in how people express body language in different cultures, one thing seems to be ubiquitous to humans: facial expressions as reactions to prime emotions, such as fear, joy, or disgust.

According to new research, mice also have facial expressions that they cannot readily control and which appear predictably under certain stimuli. This important work might help unravel the evolutionary origin of emotions.

The universality of emotional facial expressions

Charles Darwin was the first to suggest that the facial expressions of emotions are universal. In fact, this was one of the centerpieces of his theory of natural selection, proposing that emotions and their expressions were biologically innate and evolutionarily adaptive, and that similarities in them could be seen phylogenetically.

This particular idea was not validated until the 1970s, when studies showed high cross-cultural agreement in judgments of emotions in faces by people in both literate and preliterate cultures.

Over 100 modern studies published since then — which have been carried out by different researchers from different institutions using different methodologies with participants from different cultures — have converged towards the same set of results, pointing towards the universal facial expressions of at least seven emotions: anger, contempt, disgust, fear, joy, sadness, and surprise.

Of mice and facial expressions

In a new study, researchers at Max Planck Institute of Neurobiology in Germany used machine vision to study the facial expressions of mice in relation to the emotions that they experienced.

Their work showed that the facial expressions of mice were not just a reaction to the environment, but rather a reflection of their internal emotional state. The researchers were able to reliably link five emotional states to the facial expressions of mice: pleasure, disgust, nausea, pain, and fear, all of which were clearly distinguishable for the computer algorithms. 

“Mice that licked a sugar solution when they were thirsty showed a much more joyful facial expression than satiated mice,” explains Nadine Gogolla, who led the study. 

When they were offered a slightly salty solution to quench their thirst, the mice exhibited a “satisfied” facial expression, while a very salty solution produced a “disgusted” face as a response.

The researchers also scanned the brains of the mice in order to investigate how neural activity in different brain regions drives facial expressions.

When they activated certain neurons with light shined on specific brain areas known to play a role in emotional processing, the mice exhibited predictable facial expressions.

One such brain area is the insular cortex, where the emotionally-related behavior and perception of emotions are processed in both humans and animals. Neurons in the insular cortex reacted with the same strength and at exactly the same time as the mouse changed facial expression.

This is actually tremendously important research. Emotions are an incredibly important aspect of human life, but research so far has not been able to precisely identify the mechanisms responsible for all the complex feelings that we have.

“We humans may notice a subtle facial change in the mice, but we can only recognize the emotion behind it with a great deal of experience and can hardly ever determine its intensity,” says Nejc Dolensek, the study’s lead author. “With our automated face recognition system, we can now measure the intensity and nature of an emotion on a timescale of milliseconds and compare it to the neuronal activity in relevant brain areas.” 

All of these results suggest that there may be “emotion neurons”, which are dedicated to processing and expressing emotion in the body, with each sensation being controlled by a different type of neuron.

“By recording facial expressions, we can now investigate the fundamental neuronal mechanisms behind emotions in the mouse animal model,” explains Nadine Gogolla. “This is an important prerequisite for the investigation of emotions and possible disorders in their processing, such as in anxiety disorders or depression.”

The findings appeared in the journal Science.