Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of
the world. However, if initialized with knowledge of high-level subgoals and transitions …

General-purpose robots coexisting with humans in their environment must learn to relate
human language to their perceptions and actions to be useful in a range of daily tasks …

Natural language is perhaps the most flexible and intuitive way for humans to communicate
tasks to a robot. Prior work in imitation learning typically requires each task be specified with …

To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the
compositional, relational, and hierarchical structure of the world, and learn to transfer it to the …

How do humans navigate to target objects in novel scenes? Do we use the
semantic/functional priors we have built over years to efficiently search and navigate? For …

Learning is an inherently continuous phenomenon. When humans learn a new task there is
no explicit distinction between training and inference. As we learn a task, we keep learning …

Allowing humans to interactively train artificial agents to understand language instructions is
desirable for both practical and scientific reasons, but given the poor data efficiency of the …

A crucial ability of mobile intelligent agents is to integrate the evidence from multiple sensory
inputs in an environment and to make a sequence of actions to reach their goals. In this …

Vision-language navigation (VLN) tasks require an agent to follow language instructions
from a human guide to navigate in previously unseen environments using visual …

Recent work has shown that deep reinforcement-learning agents can learn to follow
language-like instructions from infrequent environment rewards. However, this places on …