One of the key arguments for building robots that have similar form factors to human beings
is that we can leverage the massive human data for training. Yet, doing so has remained …

Scaling deep learning to massive, diverse internet data has yielded remarkably general
capabilities in visual and natural language understanding and generation. However, data …

We address the task of generating temporally consistent and physically plausible images of
actions and object state transformations. Given an input image and a text prompt describing …

LLMs with visual inputs, ie, Vision Language Models (VLMs), have the capacity to process
state information as visual-textual prompts and respond with policy decisions in text. We …

Combining offline and online reinforcement learning (RL) is crucial for efficient and safe
learning. However, previous approaches treat offline and online learning as separate …

Robotic motor control necessitates the ability to predict the dynamics of environments and
interaction objects. However, advanced self-supervised pre-trained visual representations in …

The capability of intelligent models to extrapolate and comprehend changes in object states
is a crucial yet demanding aspect of AI research, particularly through the lens of human …

Learning from human demonstrations has exhibited remarkable achievements in robot
manipulation. However, the challenge remains to develop a robot system that matches …

In Model-Based Reinforcement Learning (MBRL), incorporating causal structures into
dynamics models provides agents with a structured understanding of the environments …

Large video models, pretrained on massive amounts of Internet video, provide a rich source
of physical knowledge about the dynamics and motions of objects and tasks. However …