Context-Dependent Models for Predicting and Characterizing Facial Expressiveness

machine learning
nonverbal behavior

Lin, Girard, & Morency


Citation (APA 7)

Lin, V., Girard, J. M., & Morency, L.-P. (2020). Context-dependent models for predicting and characterizing facial expressiveness. Proceedings of the 3rd Workshop on Affective Content Analysis Co-Located with the 34th AAAI Conference on Artificial Intelligence, 2614, 11–28.


In recent years, extensive research has emerged in affective computing on topics like automatic emotion recognition and determining the signals that characterize individual emotions. Much less studied, however, is expressiveness—the extent to which someone shows any feeling or emotion. Expressiveness is related to personality and mental health and plays a crucial role in social interaction. As such, the ability to automatically detect or predict expressiveness can facilitate significant advancements in areas ranging from psychiatric care to artificial social intelligence. Motivated by these potential applications, we present an extension of the BP4D+ dataset with human ratings of expressiveness and develop methods for (1) automatically predicting expressiveness from visual data and (2) defining relationships between interpretable visual signals and expressiveness. In addition, we study the emotional context in which expressiveness occurs and hypothesize that different sets of signals are indicative of expressiveness in different con-texts (e.g., in response to surprise or in response to pain). Analysis of our statistical models confirms our hypothesis. Consequently, by looking at expressiveness separately in distinct emotional contexts, our predictive models show significant improvements over baselines and achieve com-parable results to human performance in terms of correlation with the ground truth.


This paper won the Best Paper Award at the Workshop on Affective Content Analysis.