There has been an emerging paradigm shift from the era of “internet AI” to “embodied AI,”
where AI algorithms and agents no longer learn from datasets of images, videos or text …

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It
offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household …

We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation.
TDW enables simulation of high-fidelity sensory data and physical interactions between …

A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory
inputs in an environment and to make a sequence of actions to reach their goals. In this …

Stereophonic audio is an indispensable ingredient to enhance human auditory experience.
Recent research has explored the usage of visual information as guidance to generate …

Several animal species (eg, bats, dolphins, and whales) and even visually impaired humans
have the remarkable ability to perform echolocation: a biological sonar used to perceive …

Vision-language navigation (VLN) tasks require an agent to follow language instructions
from a human guide to navigate in previously unseen environments using visual …

Exploration is one of the core challenges in reinforcement learning. A common formulation
of curiosity-driven exploration uses the difference between the real future and the future …

Given only a few glimpses of an environment, how much can we infer about its entire
floorplan? Existing methods can map only what is visible or immediately apparent from …

We propose a self-supervised approach for learning to perform audio source separation in
videos based on natural language queries, using only unlabeled video and audio pairs as …