RESEARCH

HOME RESEARCH
Behavior Computing
Speech and Language
The Importance of Calibration: Rethinking Confidence and Performance of Speech Multi-label Emotion Classifiers
Abstract
The uncertainty in the modeling of emotions makes speech emotion recognition (SER) systems less reliable. An intuitive way to increase trust in SER is to reject predictions with low confidence. This approach assumes that an SER system is well calibrated, where high confident predictions are often right and low confident predictions are often wrong. Hence, it is desir- able to calibrate the confidence of SER classifiers. We eval- uate the reliability of SER systems by exploring the relation- ship between confidence and accuracy, using the expected cal- ibration error (ECE) metric. We develop a multi-label variant of the post-hoc temperature scaling (TS) method to calibrate SER systems, while preserving their accuracy. The best method combines an emotion co-occurrence weight penalty function, a class-balanced objective function, and the proposed multi-label TS calibration method. The experiments show the effectiveness of our developed multi-label calibration method in terms of ac- curacy and ECE.
Figures
Confidence versus accuracy diagrams of predictions for a model trained with LP+CB (α = 0.2) for happiness be- fore and after the multi-label TS calibration method. The grey bars are the distribution of the data based on confidence.
Confidence versus accuracy diagrams of predictions for a model trained with LP+CB (α = 0.2) for happiness be- fore and after the multi-label TS calibration method. The grey bars are the distribution of the data based on confidence.
Keywords
speech emotion recognition | confidence calibra- tion | class-balance loss | multi-label classification
Authors
Publication Date
2023/08/22
Conference
Interspeech
Interspeech 2023
Publisher
ISCA