RESEARCH

HOME RESEARCH
Multimedia Modeling
Small Group
Affect
Belongingness and Satisfaction Recognition from Physiological Synchrony with A Group-Modulated Attentive BLSTM under Small-group Conversation
Abstract
Physiological synchrony is a particular phenomenon of physiological responses during a face-face conversation. However, while many previous studies proposed various physiological synchrony measures between interlocutors in dyadic conversations, very few works on computing physiological synchrony in small groups (three or more people). Besides, belongingness and satisfaction are two critical factors for humans to decide where group they want to stay. Therefore, we want to investigate and reveal the relationship between physiological synchrony and belongingness/satisfaction under group conversation in this preliminary work. We feed the physiology ofgroup members into a designed learnable graph structure with the group-level physiological synchrony and heart-related features computed from Photoplethysmography (PPG) signals. We then devise a Group-modulated Attentive Bi-directional Long ShortTerm Memory (GGA-BLSTM) model to recognize groups' three levels ofbelongingness and satisfaction (low, middle, and high). Finally, we evaluate the proposed method on our recently collected multimodal group interaction corpus (never published before), NTUBA. The results show that (1) the models trained jointly with the grouplevel physiological synchrony and the conventional heart-related features consistently outperforms the model only trained with the conventional features, and (2) the proposed model with a Graphstructure Group-modulated Attention mechanism (GGA), GGABLSTM, performs better than the robust baseline model, the attentive BLSTM. Finally, the GGA-BLSTM achieves a good unweighted average recall (UAR) of 73.3% and 82.1% on group satisfaction and belongingness classification tasks, respectively. In further analyses, we reveal the relationships between physiological synchrony and group satisfaction/belongingness.
Figures
The overview of the proposed Graph-structure Group-modulated Attentive Bi-directional Long Short-Term Memory (GGA-BLSTM) on group satisfaction/belongingness recognition tasks through PPG features (PPG) and physiological synchrony features (△PS) computed with PPG among group members.
The overview of the proposed Graph-structure Group-modulated Attentive Bi-directional Long Short-Term Memory (GGA-BLSTM) on group satisfaction/belongingness recognition tasks through PPG features (PPG) and physiological synchrony features (△PS) computed with PPG among group members.
Keywords
small group | satisfaction | belongingness | physiological synchrony | graph attentive BLSTM
Authors
Publication Date
2021/10/18
Conference
Companion Publication of the 2021 International Conference on Multimodal Interaction
Companion Publication of the 2021 International Conference on Multimodal Interaction
DOI
10.1145/3461615.3485410
Publisher
ACM