SIMMC 2.0: A task-oriented dialog dataset for immersive multimodal conversations

S Kottur, S Moon, A Geramifard… - arxiv preprint arxiv …, 2021 - arxiv.org
Next generation task-oriented dialog systems need to understand conversational contexts
with their perceived surroundings, to effectively help users in the real-world multimodal …

Design of a competition specifically for spoken dialogue with a humanoid robot

T Minato, R Higashinaka, K Sakai, T Funayama… - Advanced …, 2023 - Taylor & Francis
Many dialogue system competitions have been held on, but no competition has been
organized specifically for spoken dialogue with humanoid robots. As the first ever such …

Affordance embeddings for situated language understanding

N Krishnaswamy, J Pustejovsky - Frontiers in artificial intelligence, 2022 - frontiersin.org
Much progress in AI over the last decade has been driven by advances in natural language
processing technology, in turn facilitated by large datasets and increased computation …

SIMMC-VR: A Task-oriented Multimodal Dialog Dataset with Situated and Immersive VR Streams

TL Wu, S Kottur, A Madotto, M Azab… - Proceedings of the …, 2023 - aclanthology.org
Building an AI assistant that can seamlessly converse and instruct humans, in a user-centric
situated scenario, requires several essential abilities:(1) spatial and temporal understanding …

Muse: A Multimodal Conversational Recommendation Dataset with Scenario-Grounded User Profiles

Z Wang, X Yang, Y Liu, S Feng, D Wang… - arxiv preprint arxiv …, 2024 - arxiv.org
Current conversational recommendation systems focus predominantly on text. However, real-
world recommendation settings are generally multimodal, causing a significant gap between …