RESEARCH

HOME RESEARCH
Behavior Computing
Mental Health
Speech and Language
Toward automating a human behavioral coding system for married couples' interactions using speech acoustic features
Abstract
Observational methods are fundamental to the study of human behavior in the behavioral sciences. For example, in the context of research on intimate relationships, psychologists' hypotheses are often empirically tested by video recording interactions of couples and manually coding relevant behaviors using standardized coding systems. This coding process can be time-consuming, and the resulting coded data may have a high degree of variability because of a number of factors (e.g., inter-evaluator differences). These challenges provide an opportunity to employ engineering methods to aid in automatically coding human behavioral data. In this work, we analyzed a large corpus of married couples' problem-solving interactions. Each spouse was manually coded with multiple session-level behavioral observations (e.g., level of blame toward other spouse), and we used acoustic speech features to automatically classify extreme instances for six selected codes (e.g., “low” vs. “high” blame). Specifically, we extracted prosodic, spectral, and voice quality features to capture global acoustic properties for each spouse and trained gender-specific and gender-independent classifiers. The best overall automatic system correctly classified 74.1% of the instances, an improvement of 3.95% absolute (5.63% relative) over our previously reported best results. We compare performance for the various factors: across codes, gender, classifier type, and feature type.
Figures
A system block diagram, illustrating the methodology taken in this paper, from pre-processing the data and extracting acoustic features to classifying extreme instances of a particular code as low/high.
A system block diagram, illustrating the methodology taken in this paper, from pre-processing the data and extracting acoustic features to classifying extreme instances of a particular code as low/high.
Keywords
Behavioral signal processing (BSP) | Couple therapy | Dyadic interaction | Human behavior analysis | Prosody | Emotion recognition
Publication Date
2013/01/01
Journal
Speech Communication 2013 Vol. 55
DOI
10.1016/j.specom.2011.12.003
Publisher
ELSEVIER