Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design
computer agents with intelligent capabilities such as understanding, reasoning, and learning …

To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions
requiring this kind of physical commonsense pose a challenge to today's natural language …

Language understanding research is held back by a failure to relate language to the
physical world it describes and to the social interactions it facilitates. Despite the incredible …

This article surveys the use of natural language in robotics from a robotics point of view. To
use human language, robots must map words to aspects of the physical world, mediated by …

A key challenge in intelligent robotics is creating robots that are capable of directly
interacting with the world around them to achieve their goals. The last decade has seen …

Exploring large corpora can generate useful discoveries but is time-consuming for humans.
We formulate a new task, D5, that automatically discovers differences between two large …

There has been a significant research interest in employing large language models to
empower intelligent robots with complex reasoning. Existing work focuses on harnessing …

Language-guided robots performing home and office tasks must navigate in and interact
with the world. Grounding language instructions against visual observations and actions to …

Seemingly simple natural language requests to a robot are generally underspecified, for
example" Can you bring me the wireless mouse?" Flat images of candidate mice may not …