Computer program can read human expressions better than humans can

We may earn a commission from links on this page.

Can you tell the difference between a genuine smile and one masking frustration? We aren't always conscious of the expressions we make in certain situations, which puts us at a disadvantage to computer programs that understand our facial expressions better than we do.

Ehsan Hoque, a graduate student in MIT's Affective Computing Group, led a new study designed to improve the way computers read and understand human faces. The team placed volunteers in front of webcams and asked them to act out various emotions, from delight to frustration. Then subjects were asked to fill out an online form designed to elicit frustration (darn thing kept deleting their data when they hit "Submit") while the webcam recorded their expressions. When asked to act out frustration, 90 percent of the subjects did not smile, but when provided with that obnoxious data-erasing form, 90 percent of them did smile.

The researchers showed also subjects videos of a delighted baby to elicit delighted smiles. While their analysis of still photos revealed little difference between delighted and frustrated smiles, the researchers did find there were distinct differences in the timing of these smiles. Delighted smiles tend to build up gradually while frustrated smiles appear suddenly and fade quickly.


As simple as this study is, it provides data that improves the capacity of computers to read emotion. When updated with this information, the group's computers can distinguish between delighted smiles and frustrated smiles better than a human can. Previous studies relied on subjects to act out emotions rather than eliciting the real thing, tainting the data and indicating that humans aren't always the best judge of our own expressions.

The hope is that, as the researchers gather more data and program the computers with more information on human expressions, these programs can help people with autism or others who have difficulty recognizing facial expressions understand the emotions flitting across other people's faces. Perhaps, though, if these programs really do understand expressions better then we do, it can help us all.


Incidentally, in the top image the smile on the right is the frustrated smile.

Photo by Hoque, et al.

Hoque's paper is available through the IEEE Computer Society.

Is that smile real or fake? [MIT News via Big Think]