Human-centered behavioral signal processing (BSP) is an emerging frontier of cross-cutting research and development field that aims at providing novel signal processing algorithms to model both the processes of complex human behavior production and subjective human behavior perception for applications of societal significance (e.g., health and education related). The advancement of BSP technologies often centers on addressing two major issues: subjectivity and time-consuming process in analyzing human behaviors.

In this 3-year proposal, we will specifically develop algorithms to analyze the socio-communicative behavior of the children with ASD during ADOS interview sessions from multimodal, i.e., audio-video, recordings. The proposal lays out a three-year research plan in the direction of developing behavior informatics to quantify atypical multimodal behaviors (atypical prosody and turn-taking behavioral coordination) and interactional multimodal behavior synchrony between clinicians and autistic child while undergoing ADOS interview sessions.

The proposal involves technical components in developing multimodal behavior feature analyses and extraction algorithms with properly contextualized machine learning framework to quantify atypicality exists both in the individual child's behavior manifestation and in the interaction coordination. At the same time, it includes a research initiative into a large-scale audio-video smart room set up collection of real patients' ADOS interview recordings.

The expected research outcome of the proposal is two folds: 1) the technical developments of these behavior informatics will be hopeful one day in transforming the current status-quo in ASD research by providing a quantification of important behavior attributes of children with ASD within the context of ADOS interaction – the overarching goal is to be able to model and control the variability in the heterogeneous behavior manifestation through mathematical computations of objective signals. 2) We anticipate the impact and contribution of such research effort can inspire additional interdisciplinary effort for human-centered research (including industry and academia): advancing existing speech and language processing technologies, sparking new capabilities of human-machine interaction, and even strengthening the bridge between behaviors science and engineering.