Unibind: Llm-augmented unified and balanced representation space to bind them all

Y Lyu, X Zheng, J Zhou, L Wang - Proceedings of the IEEE …, 2024‏ - openaccess.thecvf.com
We present UniBind a flexible and efficient approach that learns a unified representation
space for seven diverse modalities--images text audio point cloud thermal video and event …

LaMI: Large language models for multi-modal human-robot interaction

C Wang, S Hasler, D Tanneberg, F Ocker… - Extended Abstracts of …, 2024‏ - dl.acm.org
This paper presents an innovative large language model (LLM)-based robotic system for
enhancing multi-modal human-robot interaction (HRI). Traditional HRI systems relied on …

[HTML][HTML] Study of Human–Robot Interactions for Assistive Robots Using Machine Learning and Sensor Fusion Technologies

R Raj, A Kos - Electronics, 2024‏ - mdpi.com
In recent decades, the potential of robots' understanding, perception, learning, and action
has been widely expanded due to the integration of artificial intelligence (AI) into almost …

Centering the value of every modality: Towards efficient and resilient modality-agnostic semantic segmentation

X Zheng, Y Lyu, J Zhou, L Wang - European Conference on Computer …, 2024‏ - Springer
Fusing an arbitrary number of modalities is vital for achieving robust multi-modal fusion of
semantic segmentation yet remains less explored to date. Recent endeavors regard RGB …

Omnibench: Towards the future of universal omni-language models

Y Li, G Zhang, Y Ma, R Yuan, K Zhu, H Guo… - arxiv preprint arxiv …, 2024‏ - arxiv.org
Recent advancements in multimodal large language models (MLLMs) have aimed to
integrate and interpret data across diverse modalities. However, the capacity of these …

[HTML][HTML] The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for …

S Pashangpour, G Nejat - Robotics, 2024‏ - mdpi.com
The potential use of large language models (LLMs) in healthcare robotics can help address
the significant demand put on healthcare systems around the world with respect to an aging …

[HTML][HTML] Reviewing human-robot collaboration in manufacturing: Opportunities and challenges in the context of industry 5.0

M Dhanda, BA Rogers, S Hall, E Dekoninck… - Robotics and Computer …, 2025‏ - Elsevier
Abstract Industry 4.0 (I4. 0) has been characterized by the increasing use of automation,
artificial intelligence, and big data in manufacturing. It has brought different machines, tools …

Omnibind: Teach to build unequal-scale modality interaction for omni-bind of all

Y Lyu, X Zheng, D Kim, L Wang - arxiv preprint arxiv:2405.16108, 2024‏ - arxiv.org
Research on multi-modal learning dominantly aligns the modalities in a unified space at
training, and only a single one is taken for prediction at inference. However, for a real …

The Conversation is the Command: Interacting with Real-World Autonomous Robots Through Natural Language

L Nwankwo, E Rueckert - Companion of the 2024 ACM/IEEE …, 2024‏ - dl.acm.org
In recent years, autonomous agents have surged in real-world environments such as our
homes, offices, and public spaces. However, natural human-robot interaction remains a key …

Embodied AI for dexterity-capable construction Robots: DEXBOT framework

H You, T Zhou, Q Zhu, Y Ye, EJ Du - Advanced Engineering Informatics, 2024‏ - Elsevier
The construction industry is integrating robots into critical tasks at an accelerated pace, with
the aim of enhancing efficiency, safety, and productivity. However, construction tasks …