The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective

J Gratch - … Transactions of the Royal Society B, 2023 - royalsocietypublishing.org
In face-to-face interactions, parties rapidly react and adapt to each other's words,
movements and expressions. Any science of face-to-face interaction must develop …

Chalearn LAP challenges on self-reported personality recognition and non-verbal behavior forecasting during social dyadic interactions: Dataset, design, and results

C Palmero, G Barquero, JCSJ Junior… - … Social Behavior in …, 2022 - proceedings.mlr.press
This paper summarizes the 2021 ChaLearn Looking at People Challenge on Understanding
Social Behavior in Dyadic and Small Group Interactions (DYAD), which featured two tracks …

Didn't see that coming: a survey on non-verbal social human behavior forecasting

G Barquero, J Núnez, S Escalera, Z Xu… - … Social Behavior in …, 2022 - proceedings.mlr.press
Non-verbal social human behavior forecasting has increasingly attracted the interest of the
research community in recent years. Its direct applications to human-robot interaction and …

Who's next? Integrating Non-Verbal Turn-Taking Cues for Embodied Conversational Agents

J Ehret, A Bönsch, P Nossol, CA Ermert… - Proceedings of the 23rd …, 2023 - dl.acm.org
Taking turns in a conversation is a delicate interplay of various signals, which we as humans
can easily decipher. Embodied conversational agents (ECAs) communicating with humans …

MultiMediate'22: Backchannel Detection and Agreement Estimation in Group Interactions

P Müller, M Dietz, D Schiller, D Thomas… - Proceedings of the 30th …, 2022 - dl.acm.org
Backchannels, ie short interjections of the listener, serve important meta-conversational
purposes like signifying attention or indicating agreement. Despite their key role, automatic …

Multimodal Voice Activity Prediction: Turn-taking Events Detection in Expert-Novice Conversation

K Onishi, H Tanaka, S Nakamura - Proceedings of the 11th International …, 2023 - dl.acm.org
Predicting the timing of utterances in dyadic conversations is essential for achieving natural
interactions between humans and virtual agents. Since the former often use non-verbal cues …

[PDF][PDF] Entrainment analysis and prosody prediction of subsequent interlocutor's backchannels in dialogue

K Ochi, K Inoue, D Lala, T Kawahara - Proc. Interspeech 2024, 2024 - sap.ist.i.kyoto-u.ac.jp
This study investigates the characteristics of backchannels showing the entrainment to the
interlocutor's speech. The prosodic features of the dialogues of attentive listening are …

[PDF][PDF] Multimodal turn-taking model using visual cues for end-of-utterance prediction in spoken dialogue systems

F Kurata, M Saeki, S Fujie, Y Matsuyama - Proc. Interspeech 2023, 2023 - teai-waseda.jp
In this study, we propose a multimodal model for predicting the end-of-utterance probability
in spoken dialogue systems, highlighting the unique role of visual cues in addition to …

A multimodal approach for modeling engagement in conversation

A Pellet-Rostaing, R Bertrand, A Boudin… - Frontiers in Computer …, 2023 - frontiersin.org
Recently, engagement has emerged as a key variable explaining the success of
conversation. In the perspective of human-machine interaction, an automatic assessment of …

Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection

K Inoue, D Lala, G Skantze, T Kawahara - arxiv preprint arxiv:2410.15929, 2024 - arxiv.org
In human conversations, short backchannel utterances such as" yeah" and" oh" play a
crucial role in facilitating smooth and engaging dialogue. These backchannels signal …