RESEARCH | BIIC Lab - NTHU | 人本訊號運算研究室 - 國立清華大學

MENU

© Behavioral Informatics &
Interaction Computation Lab.
All rights reserved.

BIIC surroundings - SPARK

BIIC surroundings - SPARK 2

BIIC surroundings - SPARK 3

RESEARCH

RESEARCH

RESEARCH FOCUS PUBLICATIONS OPEN RESOURCE

HOME

RESEARCH

TOP ARTICLE

Multimedia Modeling・Affect

Jointly Learning from Unimodal and Multimodal-Rated Labels in Audio-Visual Emotion Recognition

Lucas Goncalves, Huang-Cheng Chou, Ali N. Salman, Chi-Chun Lee, Carlos Busso

Health Analytics・Mental Health

Using Measures of Vowel Space for Autistic Traits Characterization

Chin-Po Chen, Ho-Hsien Pan, Susan Shur-Fen Gau, Chi-Chun Lee

IEEE Transactions on Audio, Speech and Language Processing

Health Analytics・Diagnosis

Lung Cancer Prediction using Electronic Claims Records: A Transformer-based Approach

Huan-Yu Chen, Hui-Min Wang, Rob Yang, Chi-Chun Lee

IEEE Journal of Biomedical and Health Informatics

See All Publications

Health Analytics・Diagnosis

Nivolumab plus ipilimumab for potentially resectable hepatocellular carcinoma: long-term efficacy and biomarker exploration

Yih-Jyh Lin, Da-Liang Ou, Yung-Yeh Su, Chia-Lang Hsu, Chin-Fu Hsiao, Bor-Sheng Ko, San-Chi Chen, Hung-Wei Wang, Jing-Houng Wang, Yao-Ming Wu, Yung-Ming Jeng, Wei-Chen Lee, Shu-Cheng Chou, Teng-Wei Chen, Chang-Fang Chiu, Shiuan Lin, Ching-Hung Hsieh, Chi-Chun Lee, Yen-Shen Shen, Ann-Lii Cheng, Li-Tzong Chen, Chiun Hsu

dendritic cells ｜ neoadjuvant ｜ peripheral blood mononuclear cells ｜ T cell exhaustion ｜ tertiary lymphoid structure

Behavior Computing・Speech and Language

Affective Priming in Emotional Annotations and its Effect on Speech Emotion Recognition

Lus Martinez-Lucas, Ali Salman, Seong-Gyum Leem, Woan-Shiuan Chien, Shreya G Upadhyay, Chi-Chun Lee, Carlos Busso

Speech Emotion Recognition ｜ Emotional Attributes ｜ Affective Priming ｜ Emotional Annotations ｜ Affective Computing

Health Analytics・Diagnosis

Personalized Federated Learning with Fuzzy Clustering for Dysarthric Speech Recognition

Jie-Shiang Yang (Jason), Jing-Tong Tzeng, Chi-Chun Lee

Fuzzy clustering ｜ Pathological speech recognition ｜ Federated learning ｜ Feature selection ｜ Alzheimer’s disease ｜ Dysarthria ｜ Automatic speech recognition

Health Analytics・Diagnosis

Clinical Correlates of Errors in Machine Learning Diagnostic Model of Autism Spectrum Disorder: Impact of Sample Cohorts

Yen-Chin Wang, Chung-Yuan Cheng, Chi-Shin Wu, Chi-Chun Lee, Susan Shur-Fen Gau

diagnostic models ｜ error analysis ｜ machine-learning ｜ autism spectrum disorder

Health Analytics・Diagnosis

A machine learning framework for cross-institute standardized analysis of flow cytometry in differentiating acute myeloid leukemia from non-neoplastic conditions

Yu-Fen Wang, En-Ping Chu, Fong-Ci Lin, Huan-Yu Chen, Tsung-Chih Chen, Joseph Hanson, Joseph D. Tario, Kai Fu, Sara A. Monaghan, Paul K. Wallace, Chi-Chun Lee, Bor-Sheng Ko

Flow cytometry ｜ Acute myeloid leukemia ｜ Universal algorithm ｜ Machine learning

Health Analytics・Diagnosis

SocialRecNet: A Multimodal LLM-Based Framework for Assessing Social Reciprocity in Autism Spectrum Disorder

Xin-Yu Chen, Chin-Po Chen, Bo-Hao Su, Susan Shur-Fen Gau, Chi-Chun Lee

Health Analytics・Other: Computation Methods for Health

Valve Token Masked Autoencoder for Missing Recordings on Cardiac Abnormality Classification

An-Yan Chang, Jing-Tong Tzeng, Huan-Yu Chen, Chun-Hsiang Huang, Edward Pei-Chuan Huang, Chi-Chun Lee

Health Analytics・Mental Health

A Dynamic Edge-Selection Mechanism in HRV Hypergraph Learning for Improved Stress Detection

Jing-Chun Wang, Woan-Shiuan Chien, Chi-Chun Lee

Health Analytics・Mental Health

Disentangle Heart Rate Signals for Improved Stress Detection

Pin-Jhao Chen, Woan-Shiuan Chien, Chi-Chun Lee

Behavior Computing・States and Traits

Noise-Robust Speech Emotion Recognition Using Shared Self-Supervised Representations with Integrated Speech Enhancement

Jing-Tong Tzeng, Seong-Gyun Leem, Ali N. Salman, Chi-Chun Lee, Carlos Busso

Behavior Computing・States and Traits

Is It Still Fair? Investigating Gender Fairness in Cross-Corpus Speech Emotion Recognition

Shreya G Upadhyay, Woan-Shiuan Chien, Chi-Chun Lee

Behavior Computing・States and Traits

Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance

Huang-Cheng Chou, Haibin Wu, Chi-Chun Lee

Behavior Computing・States and Traits

Toward Zero-Shot Speech Emotion Recognition Using LLMs in the Absence of Target Data

Bo-Hao Su, Shreya G Upadhyay, Chi-Chun Lee

Behavior Computing・States and Traits

Mouth Articulation-Based Anchoring for Improved Cross-Corpus Speech Emotion Recognition

Shreya G Upadhyay, Ali N. Salman, Carlos Busso, Chi-Chun Lee

Health Analytics・Diagnosis

Mask Augmentation For Tumor Classification In Medical Images

Chun-Chieh Weng, Huan-Yu Chen, Jing-Tong Tzeng, Ching-Heng Lin, Po-Chih Kuo, Chi-Chun Lee

Behavior Computing・States and Traits

Improving the Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning–Based Audio Enhancement: Algorithm Development and Validation

Jing-Tong Tzeng, Jeng-Lin Li, Huan-Yu Chen, Chun-Hsiang Huang, Cheng-Yi Fan, Edward Pei-Chuan Huang, Chi-Chun Lee

clinical applicability ｜ noise robustness ｜ audio enhancement ｜ lung sound ｜ respiratory sound ｜ artificial intelligence ｜ AI

Multimedia Modeling・Affect

Jointly Learning from Unimodal and Multimodal-Rated Labels in Audio-Visual Emotion Recognition

Lucas Goncalves, Huang-Cheng Chou, Ali N. Salman, Chi-Chun Lee, Carlos Busso

Behavior Computing・Spoken Dialogs

Phonetically-Anchored Domain Adaptation for Cross-Lingual Speech Emotion Recognition

Shreya G Upadhyay, Luz Martinez-Lucas, William Katz, Carlos Busso, Chi-Chun Lee

Acoustics ｜ Affective Computing ｜ Speech Emotion Recognition ｜ Cross Lingual ｜ Emotion Recognition ｜ Adaptation Models ｜ Phonetics ｜ Linguistics ｜ Transfer Learning

Health Analytics・Mental Health

Understanding Missing Data Bias in Longitudinal Mental Stress Detection

Woan-Shiuan Chien, Chi-Chun Lee

Health Analytics・Diagnosis

In-The-Wild HRV-Based Stress Detection Using Individual-Aware Metric Learning

Jing-Chun Wang, Woan-Shiuan Chien, Huan-Yu Chen, Chi-Chun Lee

Multimedia Modeling・Affect

Stress Detection Using HRV Features Augmentation Based on Heart Rate Signal Transformation

Pin-Jhao Chen, Woan-Shiuan Chien, Huan-Yu Chen, Chi-Chun Lee

Health Analytics・Diagnosis

The unreliability of crackles: insights from a breath sound study using physicians and artificial intelligence

Chun-Hsiang Huang, Jing-Tong Tzeng, An-Yan Chang, Edward Pei-Chuan Huang, Chi-Hsin Chen, Cheng-Yi Fan, Chih-Wei Sung

Trustworthy AI・Fairness

An Inter-Speaker Fairness-Aware Speech Emotion Regression Framework

Hsing-Hang Chou, Woan-Shiuan Chien, Ya-Tse Wu, Chi-Chun Lee

Trustworthy AI・Fairness

Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition

Yi-Cheng Lin, Haibin Wu, Huang-Cheng Chou, Chi-Chun Lee, Hung-yi Lee

Trustworthy AI・Privacy

Can Modelling Inter-Rater Ambiguity Lead To Noise-Robust Continuous Emotion Predictions?

Ya-Tse Wu, Jingyao Wu, Vidhyasaharan Sethu, Chi-Chun Lee

Trustworthy AI・Privacy

A Cluster-based Personalized Federated Learning Strategy for End-to-End ASR of Dementia Patients

Wei-Tung Hsu, Chin-Po Chen, Woan-Shiuan Chien, Ya-Tse Wu, Chi-Chun Lee

Trustworthy AI・Fairness

A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition

Shreya G Upadhyay, Carlos Busso, Chi-Chun Lee

Trustworthy AI・Safety

SWiBE: A Parameterized Stochastic Diffusion Process for Noise-Robust Bandwidth Expansion

Yin-Tse Lin, Shreya G Upadhyay

Trustworthy AI・Fairness

An Investigation of Group versus Individual Fairness in Perceptually Fair Speech Emotion Recognition

Woan-Shiuan Chien, Chi-Chun Lee

Trustworthy AI・Privacy

RW-VoiceShield: Raw Waveform-based Adversarial Attack on One-shot Voice Conversion

Ching-Yu Yang, Shreya G Upadhyay, Ya-Tse Wu, Chi-Chun Lee

Behavior Computing・Speech and Language

Learning with Rater-Expanded Label Space to Improve Speech Emotion Recognition

Shreya G Upadhyay, Woan-Shiuan Chien, Bo-Hao Su, Chi-Chun Lee

speech emotion recognition ｜ multi-tasking ｜ rater subjectivity ｜ perception consistency clusters

Health Analytics・Diagnosis

Enhancing Access and Reproducibility of AML Flow Cytometry Data Analysis through Machine Learning Based Automated Cross-Panel Classification at Sample and Cell Level

Yu-Fen Wang, Yi-Syuan Li, Huan-Yu Chen, Sara A. Monaghan, Chi-Chun Lee, Bor-Sheng Ko

Health Analytics・Other: Computation Methods for Health

GaP-aug: Gamma Patch-Wise Correction Augmentation Method for Respiratory Sound Classification

An-Yan Chang, Jing-Tong Tzeng, Huan-Yu Chen, Chih-Wei Sung, Chun-Hsiang Huang, Edward Pei-Chuan Huang, Chi-Chun Lee

Gamma correction ｜ Respiratory sound classification ｜ Data augmentation ｜ Mix up

Behavior Computing・States and Traits

Balancing Speaker-Rater Fairness for Gender-Neutral Speech Emotion Recognition

Woan-Shiuan Chien, Shreya G Upadhyay, Chi-Chun Lee

Fairness ｜ Speech emotion recognition ｜ Gender neutrality ｜ Speaker-rater biases

Behavior Computing・States and Traits

Concealing Medical Condition by Node Toggling in ASR for Dementia Patients

Wei-Tung Hsu, Chin-Po Chen, Chi-Chun Lee

Automatic speech recognition ｜ Privacy-preserving machine learning (PPML) ｜ Node-cancellation ｜ Dementia

Health Analytics・Clinical Attributes

In-the-Wild Physiological-based Stress Detection Using Federated Strategy

Po-Chen Lin, Jeng-Lin Li, Woan-Shiuan Chien, Chi-Chun Lee

Stress detection ｜ Heart rate variability ｜ Federated learning ｜ Mutual learning

Health Analytics・Mental Health

Using Measures of Vowel Space for Autistic Traits Characterization

Chin-Po Chen, Ho-Hsien Pan, Susan Shur-Fen Gau, Chi-Chun Lee

Autism ｜ vowel space characteristics ｜ conversation ｜ severity assessment ｜ diagnosis

Health Analytics・Diagnosis

Lung Cancer Prediction using Electronic Claims Records: A Transformer-based Approach

Huan-Yu Chen, Hui-Min Wang, Rob Yang, Chi-Chun Lee

Deep Learning ｜ Transformer ｜ Lung Cancer ｜ Electronic Claims Records

Multimedia Modeling・Other: Media Processing for Interpretation

STELIN-US: A Spatio-Temporally Linked Neighborhood Urban Sound Database

Snehit Chunarkar, Bo-Hao Su, Chi-Chun Lee

Sound Synthesis ｜ Audio Dataset ｜ Urban Sound ｜ Connected

Behavior Computing・States and Traits

An Intelligent Infrastructure Toward Large Scale Naturalistic Affective Speech Corpora Collection

Shreya G Upadhyay, Woan-Shiuan Chien, Bo-Hao Su, Lucas Goncalves, Ya-Tse Wu, Ali N. Salman, Carlos Busso, Chi-Chun Lee

Database Consortium ｜ Affective Computing ｜ Speech Emotion Recognition

Behavior Computing・States and Traits

Analyzing the Effect of Affective Priming on Emotional Annotations

Luz Martinez-Lucas, Ali N. Salman, Seong-Gyun Leem, Shreya G Upadhyay, Chi-Chun Lee, Carlos Busso

Affective Computing ｜ Emotional Annotations ｜ Affective Priming ｜ Emotional Attributes ｜ Speech Emotion Recognition

Behavior Computing・States and Traits

The Importance of Calibration: Rethinking Confidence and Performance of Speech Multi-label Emotion Classifiers

Huang-Cheng Chou, Lucas Goncalves, Seong-Gyun Leem, Chi-Chun Lee, Carlos Busso

confidence calibra- tion ｜ class-balance loss ｜ multi-label classification ｜ speech emotion recognition

Behavior Computing・Spoken Dialogs

Speaking State Decoder with Transition Detection for Next Speaker Prediction

Shao-Hao Lu, Yun-Shao Lin, Chi-Chun Lee

next speaker prediction ｜ transition detection ｜ at-tention mechanism ｜ decoder

Behavior Computing・Speech and Language

Noise-Robust Bandwidth Expansion for 8K Speech Recordings

Yin-Tse Lin, Bo-Hao Su, Chi-Han Lin, Shih-Chan Kuo, Jyh-Shing Roger Jang, Chi-Chun Lee

Automated speech recognition ｜ Robust speech represen- tation learning ｜ Bandwidth expansion

Behavior Computing・Speech and Language

MetricAug: A Distortion Metric-Lead Augmentation Strategy for Training Noise-Robust Speech Emotion Recognizer

Ya-Tse Wu, Chi-Chun Lee

speech emotion recognition ｜ peech distortion metrics ｜ noise robustness

Behavior Computing・Deception

A Context-Constrained Sentence Modeling for Deception Detection in Real Interrogation

Ya-Tse Wu, Yuan-Ting Chang, Shao-Hao Lu, Jing-Yi Chuang, Chi-Chun Lee

Deception Detection ｜ Contextual Modeling ｜ Real Interrogation

Multimedia Modeling・Affect

An Engineering View on Emotions and Speech: From Analysis and Predictive Models to Responsible Human-Centered Applications

Chi-Chun Lee, Theodora Chaspari, Emily Mower Provost, Shrikanth S. Narayanan

Speech analysis ｜ Affect ｜ Deep learning ｜ Emotion ｜ Ethics ｜ Prosody ｜ Real-life monitoring ｜ Responsible design

Behavior Computing・States and Traits

Phonetic Anchor-Based Transfer Learning To Facilitate Unsupervised Cross-Lingual Speech Emotion Recognition

Shreya G Upadhyay, Luz Martinez-Lucas, Bo-Hao Su, Wei-Cheng Lin, Woan-Shiuan Chien, Ya-Tse Wu, William Katz, Carlos Busso, Chi-Chun Lee

transfer learning ｜ cross-lingual ｜ domain adaptation ｜ speech emotion recognition

Behavior Computing・States and Traits

Achieving Fair Speech Emotion Recognition via Perceptual Fairness

Woan-Shiuan Chien, Chi-Chun Lee

perceptual fairness ｜ fair representation ｜ rater bias ｜ speech emotion recognition

Behavior Computing・Small Group

An Interaction Process Guided Framework for Small-Group Performance Prediction

Yun-Shao Lin, Yi-Ching Liu, Chi-Chun Lee

small group interaction ｜ supervised Auto-encoder ｜ communicative functions ｜ multimodal behaviors

Behavior Computing・Spoken Dialogs

Monologue versus Conversation: Differences in Emotion Perception and Acoustic Expressivity

Woan-Shiuan Chien, Shreya G Upadhyay, Wei-Cheng Lin, Ya-Tse Wu, Bo-Hao Su, Carlos Busso, Chi-Chun Lee

speech emotion recognition ｜ emotion perception ｜ monologue ｜ conversation ｜ acoustic expression

Multimedia Modeling・Content Analysis

Romantic and Family Movie Database: Towards Understanding Human Emotion and Relationship via Genre-Dependent Movies

Po-Chien Hsu, Jeng-Lin Li, Chi-Chun Lee

perceived and induced emotions ｜ degree of being moved ｜ character relationship ｜ film genre ｜ movie database

Behavior Computing・States and Traits

An Attention-Based Method for Guiding Attribute-Aligned Speech Representation Learning

Yu-Lin Huang, Bo-Hao Su, Y.-W. Peter Hong, Chi-Chun Lee

fair ｜ attribute alignment ｜ privacy ｜ feature scoring ｜ speech representation

Behavior Computing・States and Traits

Vaccinating SER to Neutralize Adversarial Attacks with Self-Supervised Augmentation Strategy

Bo-Hao Su, Chi-Chun Lee

adversarial attacks ｜ self-supervised learning ｜ augmentation ｜ speech emotion recognition

Behavior Computing・States and Traits

Exploiting Co-occurrence Frequency of Emotions in Perceptual Evaluations To Train A Speech Emotion Classifier

Huang-Cheng Chou, Chi-Chun Lee, Carlos Busso

Speech emotion recognition ｜ emotion cooccurrence ｜ multi-class classification ｜ multi-label classification

Behavior Computing・States and Traits

Emotion-Shift Aware CRF for Decoding Emotion Sequence in Conversation

Chun-Yu Chen, Yun-Shao Lin, Chi-Chun Lee

emotion shift ｜ conditional random field ｜ conversation ｜ speech emotion recognition

Multimedia Modeling・Affect

Improving Induced Valence Recognition by Integrating Acoustic Sound Semantics in Movies

Shreya G Upadhyay, Bo-Hao Su, Chi-Chun Lee

transformer ｜ crossmodal attention ｜ induced emotion ｜ sound event detection

Multimedia Modeling・Content Analysis

Improving Multimodal Movie Scene Segmentation Using Mixture of Acoustic Experts

Meng-Han Lin, Jeng-Lin Li, Chi-Chun Lee

Movie ｜ Audio ｜ Multimodal Attention ｜ Scene Segmentation ｜ Mixture of Experts

Health Analytics・Predictive Model

A Coarse-to-Fine Pathology Patch Selection for Improving Gene Mutation Prediction in Acute Myeloid Leukemia

Chun-Chia Chiu, Jeng-Lin Li, Yu-Fen Wang, Bor-Sheng Ko, Chi-Chun Lee

Behavior Computing・States and Traits

An Enroll-to-Verify Approach for Cross-Task Unseen Emotion Class Recognition

Jeng-Lin Li, Chi-Chun Lee

prototypical loss ｜ negative margin ｜ cross-task modeling ｜ unseen class

Health Analytics・Predictive Model

A Chunking-for-Pooling Strategy for Cytometric Representation Learning for Automatic Hematologic Malignancy Classification

Jeng-Lin Li, Yun-Chun Lin, Yu-Fen Wang, Sara A. Monaghan, Bor-Sheng Ko, Chi-Chun Lee

pooling ｜ hematologic malignancy ｜ flow cytometry ｜ ensemble ｜ representation learning

Behavior Computing・Spoken Dialogs

Exploiting Annotators' Typed Description of Emotion Perception to Maximize Utilization of Ratings for Speech Emotion Recognition

Huang-Cheng Chou, Wei-Cheng Lin, Chi-Chun Lee, Carlos Busso

Multi-label learning ｜ Soft-label learning ｜ Distribution-label learning ｜ Emotion recognition

Behavior Computing・States and Traits

An Audio-Saliency Masking Transformer for Audio Emotion Classification in Movies

Ya-Tse Wu, Jeng-Lin Li, Chi-Chun Lee

emotion recognition ｜ transformer ｜ affective multimedia ｜ auditory saliency

Behavior Computing・States and Traits

Unsupervised Cross-Corpus Speech Emotion Recognition Using a Multi-Source Cycle-GAN

Bo-Hao Su, Chi-Chun Lee

speech emotion recognition ｜ data augmentation ｜ cross corpus ｜ unsupervised learning ｜ multi-sources attention

Health Analytics・Diagnosis

Federated Learning via Conditioned Mutual Learning for Alzheimer's Disease Classification on T1w MRI

Ya-Lin Huang, Hao-Chun Yang, Chi-Chun Lee

Behavior Computing・States and Traits

Learning Enhanced Acoustic Latent Representation for Small Scale Affective Corpus with Adversarial Cross Corpora Integration

Chun-Min Chang, Chi-Chun Lee

adversarial network ｜ acoustic representation ｜ speech emotion recognition ｜ cross corpus learning

Behavior Computing・Speech and Language

Deep Representation Learning for Affective Speech Signal Analysis and Processing: Preventing unwanted signal disparities

Chi-Chun Lee, Kusha Sridhar, Jeng-Lin Li, Wei-Cheng Lin, Bo-Hao Su, Carlos Busso

deep representation learning ｜ robustness ｜ generalization ｜ speech emotion recognition ｜ usability

Behavior Computing・Small Group

Self-assessed Emotion Classification from Acoustic and Physiological Features within Small-group Conversation

Woan-Shiuan Chien, Huang-Cheng Chou, Chi-Chun Lee

group interaction ｜ multi-label emotion classification ｜ Multi-modal interaction ｜ group emotion regression

Multimedia Modeling・Physiology

Belongingness and Satisfaction Recognition from Physiological Synchrony with A Group-Modulated Attentive BLSTM under Small-group Conversation

Woan-Shiuan Chien, Huang-Cheng Chou, Chi-Chun Lee

small group ｜ satisfaction ｜ belongingness ｜ graph attentive BLSTM ｜ physiological synchrony

Health Analytics・Diagnosis

A Machine Learning Approach to the Classification of Acute Leukemias and Distinction From Nonneoplastic Cytopenias Using Flow Cytometry Data

Sara A. Monaghan, Jeng-Lin Li, Yen-Chun Liu, Ming-Ya Ko, Michael Boyiadzis, Ting-Yu Chang, Yu-Fen Wang, Chi-Chun Lee, Steven H. Swerdlow, Bor-Sheng Ko

Flow cytometry ｜ Acute promyelocytic leukemia ｜ Acute myeloid leukemia ｜ B-cell lymphoblastic leukemia/lymphoma ｜ Machine learning

Behavior Computing・States and Traits

Enforcing Semantic Consistency for Cross Corpus Emotion Prediction using Adversarial Discrepancy Learning

Chun-Min Chang, Gao-Yi Chao, Chi-Chun Lee

domain adaptation ｜ semantic consistency ｜ cross corpus learning ｜ speech emotion recognition ｜ generative adversarial network

Behavior Computing・Speech and Language

An Attribute-Aligned Strategy for Learning Speech Representation

Yu-Lin Huang, Bo-Hao Su, Y.-W. Peter Hong, Chi-Chun Lee

layered dropout ｜ privacy ｜ fair ｜ attribute alignment ｜ speech representation

Behavior Computing・Deception

“Does it Matter When I Think You Are Lying?” Improving Deception Detection by Integrating Interlocutor's Judgements in Conversations

Huang-Cheng Chou, Woan-Shiuan Chien, Da-Cheng Juan, Chi-Chun Lee

deception detection ｜ human judgements ｜ dyadic conversations

Multimedia Modeling・Affect

A Media-Guided Attentive Graphical Network for Personality Recognition Using Physiology

Hao-Chun Yang, Chi-Chun Lee

physiology ｜ personality recognition ｜ affective multimedia ｜ graph convolution network

Behavior Computing・States and Traits

A Social Condition-Enhanced Network for Recognizing Power Distance using Expressive Prosody and Intrinsic Brain Connectivity

Fu-Sheng Tsai, Wei-Wen Chang, Chi-Chun Lee

prosody ｜ center-loss embedding ｜ power distance index ｜ fMRI ｜ culture dimensions

Behavior Computing・Deception

Automatic Deception Detection using Multiple Speech and Language Communicative Descriptors in Dialogs

Huang-Cheng Chou, Yi-Wen Liu, Chi-Chun Lee

Implicature classification ｜ Conversational temporal dynamics ｜ BLSTM with hierarchical attention networks ｜ Automatic deception detection

Health Analytics・General Well-being

Empowering the public during the COVID pandemic through interactive social media platform

Su-Fen Cheng, Chien-Lin Kuo, Chi-Chun Lee, Serene Hsin-Min Wei, Chu-Yu Huang

Public empowerment ｜ Live broadcasting ｜ Facebook platform ｜ COVID-19

Health Analytics・Predictive Model

Facial expression differences indicate pain improvement at the emergency department

Kuo-Cheng Wang, Chen-June Seak, Fu-Sheng Tsai, Cheng-Yu Chien, Chi-Chun Lee, Chip-Jin Ng, Bo-Cyuan Wang, Yi-Ming Weng

Pain ｜ Pain measurement ｜ Analogue pain scale ｜ Facial expression ｜ Facial recognition

Behavior Computing・Speech and Language

A Conditional Cycle Emotion Gan for Cross Corpus Speech Emotion Recognition

Bo-Hao Su, Chi-Chun Lee

conditional cycle GAN ｜ speech emotion recognition ｜ cross corpus ｜ data augmentation ｜ transfer learning

Multimedia Modeling・Content Analysis

Through the Words of Viewers: Using Comment-Content Entangled Network for Humor Impression Recognition

Huan-Yu Chen, Yun-Shao Lin, Chi-Chun Lee

speech content ｜ entangled network ｜ humor recognition ｜ viewer comments

Multimedia Modeling・Affect

From Intended to Subjective: A Conditional Tensor Fusion Network for Recognizing Self-Reported Emotion Using Physiology

Hao-Chun Yang, Chi-Chun Lee

emotion recognition ｜ feature extraction ｜ medical signal processing ｜ physiology ｜ electrocardiography

Behavior Computing・Deception

“Your Behavior Makes Me Think It Is a Lie”: Recognizing Perceived Deception using Multimodal Data in Dialog Games

Huang-Cheng Chou, Chi-Chun Lee

perceived deception detection ｜ human judgements ｜ dyadic conversations ｜ personality traits

Behavior Computing・States and Traits

Sensing with Contexts: Crying Reason Classification for Infant Care Center with Environmental Fusion

Chun-Min Chang, Huan-Yu Chen, Hsiang-Chun Chen, Chi-Chun Lee

signal classification ｜ patient care ｜ paediatrics ｜ inference mechanisms ｜ health care ｜ acoustic signal processing

Behavior Computing・Small Group

Predicting Collaborative Task Performance Using Graph Interlocutor Acoustic Network in Small Group Interaction

Shun-Chang Zhong, Bo-Hao Su, Wei Huang, Yi-Ching Liu, Chi-Chun Lee

attention mechanism ｜ personality ｜ group interaction ｜ graph convolutional network

Behavior Computing・Speech and Language

Speech Representation Learning for Emotion Recognition Using End-to-End ASR with Factorized Adaptation

Song-Lin Yeh, Yun-Shao Lin, Chi-Chun Lee

acoustic representation ｜ end-to-end ASR ｜ domain adaptation ｜ speech emotion recognition

Behavior Computing・Speech and Language

Improving Speech Emotion Recognition Using Graph Attentive Bi-Directional Gated Recurrent Unit Network

Bo-Hao Su, Chun-Min Chang, Chi-Chun Lee

speech emotion recognition ｜ graph ｜ attention mechanism ｜ recurrent neural network

Multimedia Modeling・Content Analysis

Attentive Convolutional Recurrent Neural Network Using Phoneme-Level Acoustic Representation for Rare Sound Event Detection

Shreya G Upadhyay, Bo-Hao Su, Chi-Chun Lee

convolution recurrent neural network ｜ attention ｜ automatic speech recognition ｜ sound event detection

Behavior Computing・States and Traits

Using Speaker-Aligned Graph Memory Block in Multimodally Attentive Emotion Recognition Network

Jeng-Lin Li, Chi-Chun Lee

memory network ｜ graph ｜ speaker embedding ｜ emotion recognition

Behavior Computing・States and Traits

Learning to Recognize Per-Rater's Emotion Perception Using Co-Rater Training Strategy with Soft and Hard Labels

Huang-Cheng Chou, Chi-Chun Lee

rater perception ｜ BLSTM-DNN ｜ soft label learning ｜ Speech Emotion Recognition (SER)

Multimedia Modeling・Affect

Cross Corpus Physiological-based Emotion Recognition Using a Learnable Visual Semantic Graph Convolutional Network

Woan-Shiuan Chien, Hao-Chun Yang, Chi-Chun Lee

physiology ｜ transfer learning ｜ emotion recognition ｜ affective multimedia ｜ graph convolution network

Health Analytics・Predictive Model

Predicting Performances on Processing and Memorizing East Asian Faces from Brain Activities in Face-Selective Regions: A Neurocomputational Approach

Gary C.-W. Shyi, Peter K.-H. Cheng, Shih-Tseng Tina Huang, Chi-Chun Lee, Fu-Sheng Tsai, Wan-Ting Hsieh, Becky Y.-C. Chen

face processing ｜ face memory ｜ face selective brain regions ｜ MVPA ｜ SVM classifier

Health Analytics・Diagnosis

Behavior Score-Embedded Brain Encoder Network for Improved Classification of Alzheimer Disease Using Resting State fMRI

Wan-Ting Hsieh, Jeremy Lefort-Besnard, Hao-Chun Yang, Li-Wei Kuo, Chi-Chun Lee

Health Analytics・Diagnosis

Learning a Phenotypic-Attribute Attentional Brain Connectivity Embedding for ADHD Classification using rs-fMRI

Ming-Shan Gao, Fu-Sheng Tsai, Chi-Chun Lee

Health Analytics・General Well-being

Deriving A Novel Health Index Using A Large-Scale Population Based Electronic Health Record With Deep Networks

Chen-Ying Hung, Huan-Yu Chen, Lawrence JK Wee, Chi-Chun Lee

Health Analytics・Diagnosis

A Knowledge-Reserved Distillation with Complementary Transfer for Automated FC-based Classification Across Hematological Malignancies

Jeng-Lin Li, Ting-Yu Chang, Yu-Fen Wang, Bor-Sheng Ko, Jih-Luh Tang, Chi-Chun Lee

Health Analytics・Clinical Attributes

Learning Converse-Level Multimodal Embedding to Assess Social Deficit Severity for Autism Spectrum Disorder

Chin-Po Chen, Susan Shur-Fen Gau, Chi-Chun Lee

BERT ｜ GRU ｜ ADOS ｜ autism

Behavior Computing・States and Traits

A Waveform-Feature Dual Branch Acoustic Embedding Network for Emotion Recognition

Jeng-Lin Li, Tzu-Yun Huang, Chun-Min Chang, Chi-Chun Lee

raw waveform ｜ end-to-end ｜ complementary learning ｜ acoustic representation ｜ speech emotion recognition

Behavior Computing・Spoken Dialogs

A Dialogical Emotion Decoder for Speech Emotion Recognition in Spoken Dialog

Song-Lin Yeh, Yun-Shao Lin, Chi-Chun Lee

dialogical emotion decoder ｜ conversation ｜ speech emotion recognition

Health Analytics・Diagnosis

Conditional Domain Adversarial Transfer for Robust Cross-Site ADHD Classification Using Functional MRI

Ya-Lin Huang, Wan-Ting Hsieh, Hao-Chun Yang, Chi-Chun Lee

adversarial domain adaptation ｜ multi-site transfer ｜ fMRI ｜ ADHD

Multimedia Modeling・Content Analysis

A Siamese Content-Attentive Graph Convolutional Network for Personality Recognition Using Physiology

Hao-Chun Yang, Chi-Chun Lee

personality recognition ｜ physiology ｜ graph convolution network ｜ affective multimedia

Behavior Computing・Small Group

Predicting Performance Outcome with a Conversational Graph Convolutional Network for Small Group Interactions

Yun-Shao Lin, Chi-Chun Lee

group performance ｜ graph convolutional network ｜ conversation ｜ small group interaction

Health Analytics・Diagnosis

A Multimodal Interlocutor-Modulated Attentional BLSTM for Classifying Autism Subgroups During Clinical Interviews

Yun-Shao Lin, Susan Shur-Fen Gau, Chi-Chun Lee

Behavioral signal processing ｜ autism spectrum disorder ｜ multimodal BLSTM ｜ attention mechanism

Health Analytics・Clinical Attributes

Pain versus Affect? An Investigation in the Relationship between Observed Emotional States and Self-Reported Pain

Fu-Sheng Tsai, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

Behavior Computing・Deception

Joint learning of conversational temporal dynamics and acoustic features for speech deception detection in dialog games

Huang-Cheng Chou, Yi-Wen Liu, Chi-Chun Lee

deception ｜ conversation ｜ BLSTM ｜ attention ｜ speech acoustics

Multimedia Modeling・Content Analysis

Through the Eyes of Viewers: A Comment-Enhanced Media Content Representation for TED Talks Impression Recognition

Huan-Yu Chen, Yun-Shao Lin, Chi-Chun Lee

viewer impressions ｜ TED talks ｜ cross-modal projection

Behavior Computing・Deception

Acoustic Indicators of Deception in Mandarin Daily Conversations Recorded from an Interactive Game

Chih-Hsiang Huang, Huang-Cheng Chou, Yi-Tong Wu, Chi-Chun Lee, Yi-Wen Liu

computational paralinguistics ｜ trust ｜ deception ｜ Mandarin ｜ corpus

Behavior Computing・Small Group

Predicting Group Performances Using a Personality Composite-Network Architecture During Collaborative Task

Shun-Chang Zhong, Yun-Shao Lin, Chun-Min Chang, Chi-Chun Lee

social signal processing ｜ attention mechanism ｜ personality traits ｜ group interaction

Behavior Computing・States and Traits

Using Attention Networks and Adversarial Augmentation for Styrian Dialect Continuous Sleepiness and Baby Sound Recognition

Song-Lin Yeh, Gao-Yi Chao, Bo-Hao Su, Yu-Lin Huang, Meng-Han Lin, Yin-Chun Tsai, Chi-Chun Lee

augmentation ｜ adversarial learning ｜ computational paralinguistics ｜ attention networks

Health Analytics・Clinical Attributes

Investigating the Variability of Voice Quality and Pain Levels as a Function of Multiple Clinical Parameters

Hui-Ting Hong, Jeng-Lin Li, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

age ｜ pain site ｜ voice quality ｜ gender ｜ pain

Behavior Computing・Other: Signal Modeling for Understanding

Enforcing Semantic Consistency for Cross Corpus Valence Regression from Speech Using Adversarial Discrepancy Learning

Gao-Yi Chao, Yun-Shao Lin, Chun-Min Chang, Chi-Chun Lee

cross corpus ｜ adversarial learning ｜ domain adaptation ｜ valence ｜ semantic consistency

Multimedia Modeling・Multimodal Model

Attentive to Individual: A Multimodal Emotion Recognition Network with Personalized Attention Profile

Jeng-Lin Li, Chi-Chun Lee

multimodal emotion recognition ｜ attention ｜ psycholinguistic norm ｜ personal attribute

Behavior Computing・Other: Signal Modeling for Understanding

A Dual-Complementary Acoustic Embedding Network Learned from Raw Waveform for Speech Emotion Recognition

Tzu-Yun Huang, Jeng-Lin Li, Chun-Min Chang, Chi-Chun Lee

endto-end learning ｜ raw waveform ｜ speech emotion recognition ｜ acoustic space augmentation

Health Analytics・Other: Computation Methods for Health

Improving Automatic Pain Level Recognition using Pain Site as an Auxiliary Task

Hui-Ting Hong, Jeng-Lin Li, Chun-Min Chang, Chi-Chun Lee

pain level ｜ triage ｜ multi-task learning ｜ Behavioral Signal Processing (BSP) ｜ pain site

Multimedia Modeling・Affect

Annotation Matters: A Comprehensive Study on Recognizing Intended, Self-reported, and Observed Emotion Labels using Physiology

Hao-Chun Yang, Chi-Chun Lee

annotation ｜ physiology ｜ affective computing ｜ emotion recognition ｜ mental process

Behavior Computing・Speech and Language

Attention Learning with Retrievable Acoustic Embedding of Personality for Emotion Recognition

Jeng-Lin Li, Chi-Chun Lee

multimodal emotion recognition ｜ cross corpus retrieval ｜ attention learning ｜ personality

Health Analytics・Diagnosis

A BLSTM with Attention Network for Predicting Acute Myeloid Leukemia Patient's Prognosis using Comprehensive Clinical Parameters

Chih-Chuan Lu, Jeng-Lin Li, Yu-Fen Wang, Bor-Sheng Ko, Jih-Luh Tang, Chi-Chun Lee

Health Analytics・Predictive Model

Predicting Gastrointestinal Bleeding Events from Multimodal In-Hospital Electronic Health Records Using Deep Fusion Networks

Chen-Ying Hung, Chi-Sen Chang, Jeng-Lin Li, Chi-Chun Lee

Health Analytics・Diagnosis

Learning a Cytometric Deep Phenotype Embedding for Automatic Hematological Malignancies Classification

Jeng-Lin Li, Yu-Fen Wang, Bor-Sheng Ko, Chi-Cheng Li, Jih-Luh Tang, Chi-Chun Lee

Health Analytics・Diagnosis

Improving Automatic Tremor and Movement Motor Disorder Severity Assessment for Parkinson's Disease with Deep Joint Training

Chun-Min Chang, Yu-Lin Huang, Jui-Cheng Chen, Chi-Chun Lee

learning (artificial intelligence) ｜ medical computing ｜ medical disorders ｜ gait analysis ｜ diseases

Multimedia Modeling・Audience

Learning Minimal Intra-Genre Multimodal Embedding from Trailer Content and Reactor Expressions for Box Office Prediction

Ming-Ya Ko, Jeng-Lin Li, Chi-Chun Lee

box office prediction ｜ multimedia ｜ multimodal embedding ｜ YouTube reactor ｜ movie trailer

Health Analytics・Diagnosis

Toward differential diagnosis of autism spectrum disorder using multimodal behavior descriptors and executive functions

Chin-Po Chen, Susan Shur-Fen Gau, Chi-Chun Lee

differential diagnosis ｜ executive functions ｜ multimodal behaviors descriptors ｜ autism spectrum disorder ｜ behavioral signal processing

Behavior Computing・Spoken Dialogs

An Interaction-aware Attention Network for Speech Emotion Recognition in Spoken Dialogs

Song-Lin Yeh, Yun-Shao Lin, Chi-Chun Lee

spoken dialogs ｜ attention mechanism ｜ interaction ｜ speech emotion recognition

Health Analytics・General Well-being

An Event-contrastive Connectome Network for Automatic Assessment of Individual Face Processing and Memory Ability

Wan-Ting Hsieh, Hao-Chun Yang, Fu-Sheng Tsai, Chon-Wen Shyi, Chi-Chun Lee

fMRI ｜ face processing and memory ｜ contrastive loss ｜ connectome embedding

Multimedia Modeling・Content Analysis

Learning Semantic-preserving Space Using User Profile and Multimodal Media Content from Political Social Network

Wei-Hao Chang, Jeng-Lin Li, Chi-Chun Lee

politics ｜ social media ｜ multimodal media data ｜ semantic-preserving space ｜ large-margin objective

Multimedia Modeling・Affect

An Attribute-invariant Variational Learning for Emotion Recognition Using Physiology

Hao-Chun Yang, Chi-Chun Lee

physiological representation ｜ emotion recognition ｜ variational learning ｜ psychophysiology ｜ personality

Behavior Computing・States and Traits

Every Rating Matters: Joint Learning of Subjective Labels and Individual Annotators for Speech Emotion Classification

Huang-Cheng Chou, Chi-Chun Lee

annotator modeling ｜ BLSTM ｜ speech emotion recognition ｜ soft label learning

Behavior Computing・States and Traits

Adversarially-enriched Acoustic Code Vector Learned from Out-of-context Affective Corpus for Robust Emotion Recognition

Chun-Min Chang, Chi-Chun Lee

behavioral signal processing (BSP) ｜ adversarial network ｜ emotion recognition ｜ cross corpus learning

Health Analytics・Other: Computation Methods for Health

Investigation of functional brain network reconfiguration during vocal emotional processing using graph-theoretical analysis

Shih-Yen Lin, Chi-Chun Lee, Yong-Sheng Chen, Li-Wei Kuo

vocal emotion ｜ resting state ｜ brain network ｜ graph-theoretical analysis ｜ functional magnetic resonance imaging

Health Analytics・Predictive Model

Development of an intelligent decision support system for ischemic stroke risk assessment in a population-based electronic health record database

Chen-Ying Hung, Tsuo-Hung Lan, Giia-Sheun Peng, Chi-Chun Lee

Health Analytics・Other: Computation Methods for Health

Development of a rapid and economic in vivo electrocardiogram platform for cardiovascular drug assay and electrophysiology research in adult zebrafish

Min-Hsuan Lin, Huang-Cheng Chou, Yu-Fu Chen, Wangta Liu, Chi-Chun Lee, Lawrence Yu-Min Liu, Yung-Jen Chuang

Behavior Computing・States and Traits

Learning an Arousal-Valence Speech Front-End Network using Media Data In-the-Wild for Emotion Recognition

Chih-Chuan Lu, Jeng-Lin Li, Chi-Chun Lee

speech front-end network ｜ convolutional neural network ｜ media data in-the-wild ｜ speech emotion recognition

Behavior Computing・Other: Signal Modeling for Understanding

Generating fMRI-Enriched Acoustic Vectors using a Cross-Modality Adversarial Network for Emotion Recognition

Gao-Yi Chao, Chun-Min Chang, Jeng-Lin Li, Ya-Tse Wu, Chi-Chun Lee

Cross-Modality Adversarial Network ｜ Acoustic Representation ｜ fMRI ｜ Speech Emotion Recognition

Health Analytics・Diagnosis

Clinically validated machine learning algorithm for detecting residual diseases with multicolor flow cytometry analysis in acute myeloid leukemia and myelodysplastic syndrome

Bor-Sheng Ko, Yu-Fen Wang, Jeng-Lin Li, Chi-Cheng Li, Pei-FangWeng, Szu-Chun Hsu, Hsin-An Hou, Huai-Hsuan Huang, Ming Yao, Chien-Ting Lin, Jia-Hau Liu, Cheng-Hong Tsai, Tai-Chung Huang, Shang-Ju Wu, Shang-Yi Huang, Wen-Chien Chou, Hwei-Fang Tien, Chi-Chun Lee, Jih-Luh Tang

Artificial intelligence ｜ Minimal residual disease ｜ Multicolor flow cytometry ｜ Myelodysplastic syndrome ｜ Acute myeloid leukemia

Behavior Computing・Small Group

Using Interlocutor-Modulated Attention BLSTM to Predict Personality Traits in Small Group Interaction

Yun-Shao Lin, Chi-Chun Lee

small group interaction ｜ personality traits ｜ behavioral signal processing ｜ attention mechanism ｜ social signal processing

Behavior Computing・Speech and Language

Automatic Detection of Speech Under Cold Using Discriminative Autoencoders and Strength Modeling with Multiple Sub-Dictionary Generation

Yi-Ying Kao, Hsiang-Ping Hsu, Chien-Feng Liao, Yu Tsao, Hao-Chun Yang, Jeng-Lin Li, Chi-Chun Lee, Hung-Shin Lee, Hsin-Min Wang

cold detection ｜ discriminative autoencoders ｜ deep neural networks ｜ computational paralinguistics

Behavior Computing・Spoken Dialogs

Automatic Assessment of Individual Culture Attribute of Power Distance Using a Social Context-Enhanced Prosodic Network Representation

Fu-Sheng Tsai, Hao-Chun Yang, Wei-Wen Chang, Chi-Chun Lee

prosody ｜ centerloss embedding ｜ culture attribute ｜ power distance ｜ behavioral signal processing

Behavior Computing・States and Traits

Self-Assessed Affect Recognition Using Fusion of Attentional BLSTM and Static Acoustic Features

Bo-Hao Su, Song-Lin Yeh, Ming-Ya Ko, Huan-Yu Chen, Shun-Chang Zhong, Jeng-Lin Li, Chi-Chun Lee

computational paralinguistics ｜ BLSTM ｜ affect recognition ｜ attention mechanism

Health Analytics・Clinical Attributes

An Interlocutor-Modulated Attentional LSTM for Differentiating between Subgroups of Autism Spectrum Disorder

Yun-Shao Lin, Susan Shur-Fen Gau, Chi-Chun Lee

autism spectrum disorder ｜ dyadic interaction ｜ attention mechanism ｜ behavioral signal processing (BSP)

Behavior Computing・Spoken Dialogs

Encoding Individual Acoustic Features Using Dyad-Augmented Deep Variational Representations for Dialog-level Emotion Recognition

Jeng-Lin Li, Chi-Chun Lee

variational deep embedding ｜ dyadic interaction ｜ emotion recognition ｜ feature augmentation ｜ frozen fine-tuning

Health Analytics・Clinical Attributes

Learning Conditional Acoustic Latent Representation with Gender and Age Attributes for Automatic Pain Level Recognition

Jeng-Lin Li, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

acoustic representation ｜ age and gender ｜ conditional variational autoencoder (CVAE) ｜ pain

Multimedia Modeling・Other: Media Processing for Interpretation

A Genre-Affect Relationship Network with Task-Specific Uncertainty Weighting foR Recognizing Induced Emotion in Music

Wei-Hao Chang, Jeng-Lin Li, Yun-Shao Lin, Chi-Chun Lee

homoscedastic uncertainty ｜ music ｜ induced emotion ｜ multi-task learning ｜ relationship network

Health Analytics・General Well-being

Improving Young Stroke Prediction by Learning with Active Data Augmenter in a Large-Scale Electronic Medical Claims Database

Chen-Ying Hung, Ching-Heng Lin, Chi-Chun Lee

Health Analytics・Mental Health

Learning Lexical Coherence Representation Using LSTM Forget Gate for Children with Autism Spectrum Disorder During Story-Telling

Yu-Shuo Liu, Chin-Po Chen, Susan Shur-Fen Gau, Chi-Chun Lee

autism spectrum disorder (ASD) ｜ long-short term memory neural network (LSTM) ｜ lexical coherence ｜ behavioral signal processing (BSP) ｜ story-telling

Behavior Computing・States and Traits

Integrating Perceivers Neural-Perceptual Responses Using a Deep Voting Fusion Network for Automatic Vocal Emotion Decoding

Wan-Ting Hsieh, Hao-Chun Yang, Ya-Tse Wu, Fu-Sheng Tsai, Li-Wei Kuo, Chi-Chun Lee

vocal emotion perception ｜ fMRI ｜ individual difference ｜ deep voting fusion neural net

Health Analytics・Clinical Attributes

A Triplet-Loss Embedded Deep Regressor Network for Estimating Blood Pressure Changes Using Prosodic Features

Hao-Chun Yang, Fu-Sheng Tsai, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

triplet loss embedding ｜ blood pressure ｜ prosody ｜ triage session ｜ behavioral signal processing

Behavior Computing・States and Traits

Computational Analyses of Thin-Sliced Behavior Segments in Session-Level Affect Perception

Wei-Cheng Lin, Chi-Chun Lee

thin-sliced affect perception ｜ multimodal behaviors ｜ Emotion recognition ｜ mutual information

Behavior Computing・Speech and Language

Experimental Study on Extreme Learning Machine Applications for Speech Enhancement

Tassadaq Hussain, Sabato Marco Siniscalchi, Chi-Chun Lee, Syu-Siang Wang, Yu Tsao, Wen-Hung Liao

Speech enhancement ｜ artificial neural networks ｜ extreme learning machine ｜ hierarchical extreme learning machines

Health Analytics・Clinical Attributes

Embedding stacked bottleneck vocal features in a LSTM architecture for automatic pain level classification during emergency triage

Fu-Sheng Tsai, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

Behavior Computing・States and Traits

NNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus

Huang-Cheng Chou, Wei-Cheng Lin, Lien-Chiang Chang, Chyi-Chang Li, Hsi-Pin Ma, Chi-Chun Lee

electrocardiography ｜ emotion recognition ｜ human computer interaction ｜ interactive systems

Behavior Computing・Speech and Language

A bootstrapped multi-view weighted Kernel fusion framework for cross-corpus integration of multimodal emotion recognition

Chun-Min Chang, Bo-Hao Su, Shih-Chen Lin, Jeng-Lin Li, Chi-Chun Lee

emotion recognition ｜ sensor fusion ｜ speech recognition ｜ video signal processing

Behavior Computing・Speech and Language

Toward Automating Oral Presentation Scoring During Principal Certification Program Using Audio-Video Low-Level Behavior Profiles

Shan-Wen Hsiao, Hung-Ching Sun, Ming-Chuan Hsieh, Ming-Hsueh Tsai, Yu Tsao, Chi-Chun Lee

Behavioral signal processing (BSP) ｜ educational research ｜ multimodal signal processing ｜ oral presentation

Health Analytics・Mental Health

Signal Processing and Machine Learning for Mental Health Research and Clinical Applications

Daniel Bone, Chi-Chun Lee, Theodora Chaspari, James Gibson, Shrikanth Narayanan

Behavior Computing・States and Traits

Modeling Perceivers Neural-Responses Using Lobe-Dependent Convolutional Neural Network to Improve Speech Emotion Recognition

Ya-Tse Wu, Yu-Hsien Liao, Chi-Chun Lee

speech emotion recognition ｜ affective computing, fMRI ｜ convolutional neural network (CNN)

Behavior Computing・States and Traits

Deriving Dyad-Level Interaction Representation Using Interlocutors Structural and Expressive Multimodal Behavior Features

Yun-Shao Lin, Chi-Chun Lee

face-to-face interaction ｜ multimodal behaviors ｜ dyad-level affect ｜ affect recognition

Health Analytics・Diagnosis

Computing Multimodal Dyadic Behaviors During Spontaneous Diagnosis Interviews Toward Automatic Categorization of Autism Spectrum Disorder

Chin-Po Chen, Xian-Hong Tseng, Susan Shur-Fen Gau, Chi-Chun Lee

dyadic interaction ｜ autism spectrum disorder ｜ behavioral signal processing (BSP) ｜ multimodal behaviors

Behavior Computing・Small Group

Initiating and Maintaining Collaborations and Facilitating Understanding in Interdisciplinary Group Research

Stephenson J. Beck, Annika L. Meinecke, Yoichi Matsuyama, Chi-Chun Lee

interdisciplinary ｜ collaboration ｜ computer science ｜ group science

Health Analytics・Predictive Model

Comparing deep neural network and other machine learning algorithms for stroke prediction in a large-scale population-based electronic medical claims database

Chen-Ying Hung, Wei-Chen Chen, Po-Tsun Lai, Ching-Heng Lin, Chi-Chun Lee

Behavior Computing・States and Traits

Fusion of multiple emotion perspectives: Improving affect recognition through integrating cross-lingual emotion information

Chun-Min Chang, Chi-Chun Lee

multi-task learning ｜ affective computing ｜ cross language ｜ speech emotion recognition

Behavior Computing・States and Traits

Minimization of Regression and Ranking Losses with Shallow Neural Networks on Automatic Sincerity Evaluation

Hung-Shin Lee, Yu Tsao, Chi-Chun Lee, Hsin-Min Wang, Wei-Cheng Lin, Wei-Chen Chen, Shan-Wen Hsiao, Shyh-Kang Jeng

shallow neural networks ｜ degree of sincerity ｜ regression, ranking ｜ computational paralinguistics

Behavior Computing・Other: Signal Modeling for Understanding

Enhancement of Automatic Oral Presentation Assessment System Using Latent N-Grams Word Representation and Part-of-Speech Information

Wen-Yu Huang, Shan-Wen Hsiao, Hung-Ching Sun, Ming-Chuan Hsieh, Ming-Hsueh Tsai, Chi-Chun Lee

educational research ｜ multimodal signal processing ｜ behavioral signal processing ｜ natural language processing

Health Analytics・Other: Computation Methods for Health

Toward Development and Evaluation of Pain Level-Rating Scale for Emergency Triage based on Vocal Characteristics and Facial Expressions

Fu-Sheng Tsai, Ya-Ling Hsu, Wei-Chen Chen, Yi-Ming Weng, Chip-Jin Ng, Chi-Chun Lee

facial expressions ｜ triage ｜ pain scale ｜ vocal characteristics ｜ behavioral signal processing (BSP)

Behavior Computing・States and Traits

A thin-slice perception of emotion? An information theoretic-based framework to identify locally emotion-rich behavior segments for global affect recognition

Wei-Cheng Lin, Chi-Chun Lee

emotion recognition ｜ behavioral signal processing ｜ thin-slice theory ｜ multimodal signal processing

Behavior Computing・States and Traits

A Gaussian mixture regression approach toward modeling the affective dynamics between acoustically-derived vocal arousal score (VC-AS) and internal brain fMRI bold signal response

, Yu-Hsien Liao, Heng-Tai Jan, Li-Wei Kuo, Chi-Chun Lee

fMRI ｜ vocal arousal score ｜ Gaussian Mixture Regression ｜ behavioral signal processing (BSP)

Behavior Computing・States and Traits

An analysis of the relationship between signal-derived vocal arousal score and human emotion production and perception

Chi-Chun Lee, Daniel Bone, Shrikanth Narayanan

vocal arousal rating ｜ affective perception ｜ affective production

Behavior Computing・Other: Signal Modeling for Understanding

A multimodal approach for automatic assessment of school principals' oral presentation during pre-service training program

Shan-Wen Hsiao, Hung-Ching Sun, Ming-Chuan Hsieh, Ming-Hsueh Tsai, Hsin-Chih Lin, Chi-Chun Lee

oral presentation ｜ multimodal signal processing ｜ education research ｜ behavioral signal processing (BSP)

Behavior Computing・Other: Signal Modeling for Understanding

Multimodal arousal rating using unsupervised fusion technique

Wei-Chen Chen, Po-Tsun Lai, Yu Tsao, Chi-Chun Lee

arousal rating ｜ affective computing ｜ behavioral signal processing ｜ multimodal signal processing

Behavior Computing・States and Traits

The USC CreativeIT database of multimodal dyadic interactions: from speech and full body motion capture to continuous emotional annotations

Angeliki Metallinou, Zhaojun Yang, Chi-Chun Lee, Carlos Busso, Sharon Carnicke, Shrikanth Narayanan

Motion capture system ｜ Theatrical improvisations ｜ Multimodal database ｜ Continuous emotion

Health Analytics・Diagnosis

Applying machine learning to facilitate autism diagnostics: pitfalls and promises

Daniel Bone, Matthew S. Goodwin, Matthew P. P. Black, Chi-Chun Lee, Kartik Audhkhasi, Shrikanth Narayanan

Machine learning ｜ Autism diagnostic interview ｜ Autism diagnostic observation schedule ｜ Signal processing ｜ Autism ｜ Diagnosis

Behavior Computing・Speech and Language

An Investigation of Vocal Arousal Dynamics in Child-Psychologist Interactions using Synchrony Measures and a Conversation-based Model

Daniel Bone, Chi-Chun Lee, Alexandros Potamianos, Shrikanth Narayanan

interaction ｜ synchrony ｜ vocal arousal ｜ autism spectrum disorders

Behavior Computing・States and Traits

Ensemble of Machine Learning Algorithms for Cognitive and Physical Speaker Load Detection

How Jing, Ting-Yao Hu, Hung-Shin Lee, Wei-Chen Chen, Chi-Chun Lee, Yu Tsao, Hsin-Min Wang

Neural Network ｜ Cognitive Load Detection ｜ Physical Load Detection ｜ Classification Models

Behavior Computing・Spoken Dialogs

The Psychologist as an Interlocutor in Autism Spectrum Disorder Assessment: Insights From a Study of Spontaneous Prosody

Daniel Bone, Chi-Chun Lee, Matthew P. Black, Marian E. Williams, Sungbok Lee, Pat Levitt, Shrikanth Narayanan

social communication ｜ children; prosody ｜ autism spectrum disorder ｜ assessment ｜ dyadic interaction

Behavior Computing・States and Traits

Robust Unsupervised Arousal Rating: A Rule-Based Framework with Knowledge-Inspired Vocal Features

Daniel Bone, Chi-Chun Lee, Shrikanth Narayanan

cross-corpora classification ｜ knowledge-inspired features ｜ rule-based rating ｜ activation ｜ arousal ｜ continuous affect tracking

Behavior Computing・States and Traits

Computing vocal entrainment: A signal-derived PCA-based quantification scheme with application to affect analysis in married couple interactions

Chi-Chun Lee, Athanasios Katsamanis, Matthew P. Black, Brian R. Baucom, Andrew Christensen, Panayiotis G. Georgiou, Shrikanth Narayanan

Factorial Hidden Markov Model ｜ Behavioral signal processing (BSP) ｜ Couple therapy ｜ Principal component analysis (PCA) ｜ Interaction synchrony ｜ Affect recognition ｜ Entrainment ｜ Dyadic interaction

Behavior Computing・Speech and Language

Acoustic-Prosodic, Turn-taking, and Language Cues in Child-Psychologist Interactions for Varying Social Demand

Daniel Bone, Chi-Chun Lee, Theodora Chaspari, Matthew P. Black, Marian E. Williams, Sungbok Lee, Pat Levitt, Shrikanth Narayanan

turn-taking ｜ social reciprocity ｜ atypical prosody ｜ autism spectrum disorders ｜ language cues

Behavior Computing・Other: Signal Modeling for Understanding

Analyzing Eye-Voice Coordination in Rapid Automatized Naming

Daniel Bone, Chi-Chun Lee, Vikram Ramanarayanan, Shrikanth Narayanan, Renske S. Hoedemaker, Peter C. Gordon

eye-tracking ｜ reading strategies ｜ Rapid Automatized Naming (RAN) ｜ multi-modal ｜ production-perception

Behavior Computing・Spoken Dialogs

Assessment of a child’s engagement using sequence model based features

Rahul Gupta, Chi-Chun Lee, Sungbok Lee, Shrikanth Narayanan

child engagement ｜ Behavior signal processing ｜ sequence model

Behavior Computing・States and Traits

Head motion synchrony and its correlation to affectivity in dyadic interactions

Bo Xiao, Panayiotis G. Georgiou, Chi-Chun Lee, Brian Baucom, Shrikanth Narayanan

Linear prediction ｜ Entrainment ｜ Behavioral signal processing ｜ Synchrony ｜ Head motion ｜ Gaussian mixture model

Behavior Computing・Spoken Dialogs

Using physiology and language cues for modeling verbal response latencies of children with ASD

Theodora Chaspari, Daniel Bone, James Gibson, Chi-Chun Lee, Shrikanth Narayanan

Autism Spectrum Disorders ｜ Language modeling ｜ Electrodermal Activity ｜ Verbal response latency ｜ Generalized linear regression

Behavior Computing・Spoken Dialogs

Toward automating a human behavioral coding system for married couples' interactions using speech acoustic features

Matthew P. P. Black, Athanasios Katsamanis, Brian R. Baucom, Chi-Chun Lee, Adam C Lammert, Andrew Christensen, Panayiotis G. Georgiou, Shrikanth Narayanan

Prosody ｜ Human behavior analysis ｜ Dyadic interaction ｜ Couple therapy ｜ Behavioral signal processing (BSP) ｜ Emotion recognition

Behavior Computing・States and Traits

Using measures of vocal entrainment to inform outcome-related behaviors in marital conflicts

Chi-Chun Lee, Athanasios Katsamanis, Brian R. Baucom, Panayiotis G. Georgiou, Shrikanth Narayanan

Behavior Computing・States and Traits

Acoustical analysis of engagement behavior in children

Rahul Gupta, Chi-Chun Lee, Daniel Bone, Agata Rozga, Sungbok Lee, Shrikanth Narayanan

Rapid-ABC ｜ Child engagement ｜ autism

Behavior Computing・Spoken Dialogs

Spontaneous-speech acoustic-prosodic features of children with autism and the interacting psychologist

Daniel Bone, Matthew P. Black, Chi-Chun Lee, Marian E. Williams, Pat Levitt, Sungbok Lee, Shrikanth Narayanan

psychologist ｜ intonation ｜ autism spectrum disorder ｜ atypical prosody ｜ voice quality ｜ ADOS

Behavior Computing・Other: Signal Modeling for Understanding

A robust unsupervised arousal rating framework using prosody with cross-corpora evaluation

Daniel Bone, Chi-Chun Lee, Shrikanth Narayanan

inter-rater reliability ｜ knowledge-based ｜ unsupervised ｜ activation ｜ arousal rating ｜ cross-corpora

Health Analytics・Mental Health

Interplay between verbal response latency and physiology of children with autism during ECA interactions

Theodora Chaspari, Chi-Chun Lee, Shrikanth Narayanan

verbal response latency ｜ affective state ｜ cognitive and social activity ｜ autism ｜ Electrodermal response

Behavior Computing・States and Traits

Based on isolated saliency or causal integration? toward a better understanding of human annotation process using multiple instance learning and sequential probability ratio test

Chi-Chun Lee, Athanasios Katsamanis, Panayiotis G. Georgiou, Shrikanth Narayanan

sequential probability ratio test ｜ behavior annotation, perception ｜ observational coding ｜ multiple instance learning

Behavior Computing・States and Traits

Classification of Emotional Content of Sighs in Dyadic Human Interactions

Rahul Gupta, Chi-Chun Lee, Shrikanth Narayanan

support vector machine ｜ multimodal fusion ｜ nonverbal vocalizations

Behavior Computing・States and Traits

Emotion recognition using a hierarchical binary decision tree approach

Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, Shrikanth Narayanan

Support Vector Machine ｜ Emotion recognition ｜ Hierarchical structure ｜ Bayesian Logistic Regression

Behavior Computing・States and Traits

Affective state recognition in married couples' interactions using PCA-based vocal entrainment measures with multiple instance learning

Chi-Chun Lee, Athanasios Katsamanis, Matthew P. Black, Brian R. Baucom, Panayiotis G. Georgiou, Shrikanth Narayanan

behavioral signal processing (BSP) ｜ couple's therapy ｜ vocal entrainment ｜ multiple instance learning ｜ affective recognition

Health Analytics・Mental Health

Analyzing the nature of ECA interactions in children with autism

Emily Mower, Chi-Chun Lee, James Gibson, Theodora Chaspari, Marian E. Williams, Shrikanth Narayanan

autism ｜ audio-video recording ｜ multimodal interface ｜ Embodied conversational agent ｜ children’s speech

Behavior Computing・Spoken Dialogs

An Analysis of PCA-based Vocal Entrainment Measures in Married Couples' Affective Spoken Interactions

Chi-Chun Lee, Athanasios Katsamanis, Matthew P. Black, Brian R. Baucom, Panayiotis G. Georgiou, Shrikanth Narayanan

couples therapy ｜ behavioral signal processing ｜ principal component analysis ｜ vocal entrainment

Behavior Computing・States and Traits

Automatic Classification of Married Couples' Behavior using Audio Features

Matthew P. Black, Athanasios Katsamanis, Chi-Chun Lee, Adam C. Lammert, Brian R. Baucom, Andrew Christensen, Panayiotis G. Georgiou, Shrikanth Narayanan

couples therapy ｜ human behavior analysis ｜ behavioral signal processing ｜ prosody ｜ emotion recognition

Behavior Computing・Spoken Dialogs

Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples

Chi-Chun Lee, Matthew P. Black, Athanasios Katsamanis, Adam C Lammert, Brian R. Baucom, Andrew Christensen, Panayiotis G. Georgiou, Shrikanth Narayanan

prosody ｜ entrainment ｜ behavior signal processing ｜ affect ｜ couple therapy ｜ interaction synchrony

Behavior Computing・States and Traits

The USC CreativeIT Database: A Multimodal Database of Theatrical Improvisation

Angeliki Metallinou, Chi-Chun Lee, Carlos Busso, Sharon Carnicke, Shrikanth Narayanan

Behavior Computing・Other: Signal Modeling for Understanding

Predicting interruptions in dyadic spoken interactions

Chi-Chun Lee, Shrikanth Narayanan

Interruption ｜ Dyadic Interaction ｜ Hidden Conditional Field ｜ Prediction

Behavior Computing・States and Traits

Interpreting ambiguous emotional expressions

Emily Mower, Angeliki Metallinou, Chi-Chun Lee, Abe Kazemzadeh, Carlos Busso, Sungbok Lee, Shrikanth Narayanan

Behavior Computing・Other: Signal Modeling for Understanding

Emotion recognition using a hierarchical binary decision tree approach

Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, Shrikanth Narayanan

Behavior Computing・Spoken Dialogs

Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions

Chi-Chun Lee, Carlos Busso, Sungbok Lee, Shrikanth Narayanan

emotion recognition ｜ mutual influence ｜ Dynamic Bayesian Network ｜ dyadic interaction

Behavior Computing・States and Traits

IEMOCAP: interactive emotional dyadic motion capture database

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N. Chang, Sungbok Lee, Shrikanth Narayanan

Dyadic interaction ｜ Emotion ｜ Emotional assessment ｜ Motion capture system ｜ Audio-visual database

Behavior Computing・Spoken Dialogs

An analysis of multimodal cues of interruption in dyadic spoken interactions

Chi-Chun Lee, Sungbok Lee, Shrikanth Narayanan

interruption ｜ multimodal ｜ turn taking ｜ dyadic interaction

Behavior Computing・States and Traits

Improving the Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning–Based Audio Enhancement: Algorithm Development and Validation

Jing-Tong Tzeng, Jeng-Lin Li, Huan-Yu Chen, Chun-Hsiang Huang, Chi-Hsin Chen, Cheng-Yi Fan, Edward Pei-Chuan Huang, Chi-Chun Lee

respiratory sound ｜ lung soun ｜ audio enhancement ｜ noise robustness ｜ clinical applicability ｜ artificial intelligence ｜ AI

Behavior Computing・States and Traits

Improving the Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning–Based Audio Enhancement: Algorithm Development and Validation

Jing-Tong Tzeng, Jeng-Lin Li, Huan-Yu Chen, Chun-Hsiang Huang, Chi-Hsin Chen, Cheng-Yi Fan, Edward Pei-Chuan Huang, Chi-Chun Lee

respiratory sound ｜ lung soun ｜ audio enhancement ｜ noise robustness ｜ clinical applicability ｜ artificial intelligence ｜ AI