RESEARCH

HOME RESEARCH
Multimedia Modeling
Affect
Other: Signal Modeling for Understanding
Physiology
Annotation Matters: A Comprehensive Study on Recognizing Intended, Self-reported, and Observed Emotion Labels using Physiology
Abstract
Studies have shown that the formation of emotion as self-awareness and cognitive appraisal process is complicated and can lead to idiosyncratic differences. Subject’s self emotion evaluation process could be biased due to factors of environment, personal experience, and one’s own cognitive ability, and the true affective state may be neglected (un-noticeable) due to an unconscious mental process. In this work, we present a comprehensive study to investigate the emotion recognition accuracy obtained using physiology with respect to different annotation schemes, i.e., intended, self-reported, and observed emotion labels. We found that when performing recognition across these three different labeling schemes using the same physiological parameters, the accuracy of the self-reported emotion labels results in about 10.3% and 3.1% drop when compared to two other annotation schemes. It indicates that self-assessed emotion labels may be noisier and induces a larger mismatch with respect to the affectstimulated physiological responses. Further analysis shows that the electrodermal activity signal has the highest recognition rate with respect to the intended emotion of the stimuli. Finally, our error analysis reveals that there may exist a bias in the selfannotated label that is conditioned on the intended stimuli’s valence polarity.
Figures
The distribution of SHAP scores obtained for each physiological modalities. The vertical axis refers to the absolute SHAP score while the horizontal axis maps to the feature dimensions to each physiology. The horizontal black line indicates the cut-off threshold at 0.025.
The distribution of SHAP scores obtained for each physiological modalities. The vertical axis refers to the absolute SHAP score while the horizontal axis maps to the feature dimensions to each physiology. The horizontal black line indicates the cut-off threshold at 0.025.
Keywords
emotion recognition | annotation | physiology | affective computing | mental process
Authors
Hao-Chun Yang Chi-Chun Lee
Publication Date
2019/09/03
Conference
International Conference on Affective Computing and Intelligent Interaction (ACII)
2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
DOI
10.1109/acii.2019.8925516
Publisher
IEEE