Real-world robot applications of foundation models: A review

K Kawaharazuka, T Matsushima… - Advanced …, 2024 - Taylor & Francis
Recent developments in foundation models, like Large Language Models (LLMs) and Vision-
Language Models (VLMs), trained on extensive data, facilitate flexible application across …

Large language models for robotics: Opportunities, challenges, and perspectives

J Wang, Z Wu, Y Li, H Jiang, P Shu, E Shi, H Hu… - arxiv preprint arxiv …, 2024 - arxiv.org
Large language models (LLMs) have undergone significant expansion and have been
increasingly integrated across various domains. Notably, in the realm of robot task planning …

Open x-embodiment: Robotic learning datasets and rt-x models

A O'Neill, A Rehman, A Gupta, A Maddukuri… - arxiv preprint arxiv …, 2023 - arxiv.org
Large, high-capacity models trained on diverse datasets have shown remarkable successes
on efficiently tackling downstream applications. In domains from NLP to Computer Vision …

Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration0

A O'Neill, A Rehman, A Maddukuri… - … on Robotics and …, 2024 - ieeexplore.ieee.org
Large, high-capacity models trained on diverse datasets have shown remarkable successes
on efficiently tackling downstream applications. In domains from NLP to Computer Vision …

Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators

P Wu, Y Shentu, Z Yi, X Lin… - 2024 IEEE/RSJ …, 2024 - ieeexplore.ieee.org
Humans can teleoperate robots to accomplish complex manipulation tasks. Imitation
learning has emerged as a powerful framework that leverages human teleoperated …

Aloha unleashed: A simple recipe for robot dexterity

TZ Zhao, J Tompson, D Driess, P Florence… - arxiv preprint arxiv …, 2024 - arxiv.org
Recent work has shown promising results for learning end-to-end robot policies using
imitation learning. In this work we address the question of how far can we push imitation …

Fmb: a functional manipulation benchmark for generalizable robotic learning

J Luo, C Xu, F Liu, L Tan, Z Lin, J Wu… - … Journal of Robotics …, 2023 - journals.sagepub.com
In this paper, we propose a real-world benchmark for studying robotic learning in the context
of functional manipulation: a robot needs to accomplish complex long-horizon behaviors by …

Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot

HS Fang, H Fang, Z Tang, J Liu, C Wang… - … on Robotics and …, 2024 - ieeexplore.ieee.org
A key challenge for robotic manipulation in open domains is how to acquire diverse and
generalizable skills for robots. Recent progress in one-shot imitation learning and robotic …

Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers

H Ha, Y Gao, Z Fu, J Tan, S Song - arxiv preprint arxiv:2407.10353, 2024 - arxiv.org
We introduce UMI-on-Legs, a new framework that combines real-world and simulation data
for quadruped manipulation systems. We scale task-centric data collection in the real world …

Generative camera dolly: Extreme monocular dynamic novel view synthesis

B Van Hoorick, R Wu, E Ozguroglu, K Sargent… - … on Computer Vision, 2024 - Springer
Accurate reconstruction of complex dynamic scenes from just a single viewpoint continues to
be a challenging task in computer vision. Current dynamic novel view synthesis methods …