[HTML][HTML] Large language models for robotics: Opportunities, challenges, and perspectives

J Wang, E Shi, H Hu, C Ma, Y Liu, X Wang… - Journal of Automation …, 2024 - Elsevier
Large language models (LLMs) have undergone significant expansion and have been
increasingly integrated across various domains. Notably, in the realm of robot task planning …

Rt-1: Robotics transformer for real-world control at scale

A Brohan, N Brown, J Carbajal, Y Chebotar… - arxiv preprint arxiv …, 2022 - arxiv.org
By transferring knowledge from large, diverse, task-agnostic datasets, modern machine
learning models can solve specific downstream tasks either zero-shot or with small task …

Perceiver-actor: A multi-task transformer for robotic manipulation

M Shridhar, L Manuelli, D Fox - Conference on Robot …, 2023 - proceedings.mlr.press
Transformers have revolutionized vision and natural language processing with their ability to
scale with large datasets. But in robotic manipulation, data is both limited and expensive …

Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions

Y Chebotar, Q Vuong, K Hausman… - … on Robot Learning, 2023 - proceedings.mlr.press
In this work, we present a scalable reinforcement learning method for training multi-task
policies from large offline datasets that can leverage both human demonstrations and …

Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration0

A O'Neill, A Rehman, A Maddukuri… - … on Robotics and …, 2024 - ieeexplore.ieee.org
Large, high-capacity models trained on diverse datasets have shown remarkable successes
on efficiently tackling downstream applications. In domains from NLP to Computer Vision …

[PDF][PDF] Vima: General robot manipulation with multimodal prompts

Y Jiang, A Gupta, Z Zhang, G Wang… - arxiv preprint …, 2022 - authors.library.caltech.edu
Prompt-based learning has emerged as a successful paradigm in natural language
processing, where a single general-purpose language model can be instructed to perform …

Enhancing autonomous system security and resilience with generative AI: A comprehensive survey

M Andreoni, WT Lunardi, G Lawton, S Thakkar - IEEE Access, 2024 - ieeexplore.ieee.org
This survey explores the transformative role of Generative Artificial Intelligence (GenAI) in
enhancing the trustworthiness, reliability, and security of autonomous systems such as …

Vima: Robot manipulation with multimodal prompts

Y Jiang, A Gupta, Z Zhang, G Wang, Y Dou, Y Chen… - 2023 - openreview.net
Prompt-based learning has emerged as a successful paradigm in natural language
processing, where a single general-purpose language model can be instructed to perform …

[PDF][PDF] Open x-embodiment: Robotic learning datasets and rt-x models

Q Vuong, S Levine, HR Walke, K Pertsch… - … for Scalable Skill …, 2023 - openreview.net
Large, high-capacity models trained on diverse datasets have shown remarkable successes
on efficiently tackling downstream applications. In domains from NLP to Computer Vision …

Baku: An efficient transformer for multi-task policy learning

S Haldar, Z Peng, L Pinto - arxiv preprint arxiv:2406.07539, 2024 - arxiv.org
Training generalist agents capable of solving diverse tasks is challenging, often requiring
large datasets of expert demonstrations. This is particularly problematic in robotics, where …