Emotion is an internal state of human that plays a critical role in our daily life such that it affects all aspect of individual’s behaviors, personality, actions, and his/her social connections. Computing emotion using data (objectively with scale) to understand humans scientifically and derive next-generation human-centered applications (reliably with novel algorithm) has sparked a tremendous interest both in field of psychology and technical field of affective computing (give rise to the term of `Emotion AI’ with the recent use of deep learning techniques). While there has been a growing interest in developing and deploying emotion AI technology, there still exists major technical proliferation bottlenecks in the wide adoption of emotion AI technology into life. In this 4 year proposal, we focus on the two core emotion AI modeling challenging dimensions include affective signal modeling and affective application popularization. Our goal is to develop a series of computational methods to achieve the following three major modeling goals:

1) Robustness: How do we learn multimodal affective-signal representations that can be robust against variable settings of both signal acquisition and the human idiosyncratic nature of emotion modulation and perception to achieve robustness?
2) Generalization: How do we learn multimodal affective-signal representations that can handle source-target domain mismatch in cross-context application scenarios to achieve generalization?
3) Usability: How do we learn multimodal affective-signal representations that can be practical during deployment and handle privacy and ethical concerns to achieve usability

At the same time, as part of the proposal, we plan to collect two additional large affective signal corpora. Our proposal would contribute on two fronts: deepening scientific understanding and grounding affect-related applications. There are four anticipated impact of our proposal: toward quantitative understanding of human emotion, advancing computational algorithms, releasing database with global community connection, and contributing to the emotion AI industry. Specifically, we will advance affective modeling algorithms to achieve proliferation, release new database to connect with the global community, provide interdisciplinary training to students, and also devise an opensource and free platform for public engagement of emotion AI algorithms with multimodal data processing pipelines.