A survey on offline reinforcement learning: Taxonomy, review, and open problems

RF Prudencio, MROA Maximo… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
With the widespread adoption of deep learning, reinforcement learning (RL) has
experienced a dramatic increase in popularity, scaling to previously intractable problems …

Transferring policy of deep reinforcement learning from simulation to reality for robotics

H Ju, R Juan, R Gomez, K Nakamura… - Nature Machine …, 2022 - nature.com
Deep reinforcement learning has achieved great success in many fields and has shown
promise in learning robust skills for robot control in recent years. However, sampling …

Efficient online reinforcement learning with offline data

PJ Ball, L Smith, I Kostrikov… - … Conference on Machine …, 2023 - proceedings.mlr.press
Sample efficiency and exploration remain major challenges in online reinforcement learning
(RL). A powerful approach that can be applied to address these issues is the inclusion of …

Cal-ql: Calibrated offline rl pre-training for efficient online fine-tuning

M Nakamoto, S Zhai, A Singh… - Advances in …, 2023 - proceedings.neurips.cc
A compelling use case of offline reinforcement learning (RL) is to obtain a policy initialization
from existing datasets followed by fast online fine-tuning with limited interaction. However …

Online decision transformer

Q Zheng, A Zhang, A Grover - international conference on …, 2022 - proceedings.mlr.press
Recent work has shown that offline reinforcement learning (RL) can be formulated as a
sequence modeling problem (Chen et al., 2021; Janner et al., 2021) and solved via …

Mildly conservative q-learning for offline reinforcement learning

J Lyu, X Ma, X Li, Z Lu - Advances in Neural Information …, 2022 - proceedings.neurips.cc
Offline reinforcement learning (RL) defines the task of learning from a static logged dataset
without continually interacting with the environment. The distribution shift between the …

Pessimistic bootstrap** for uncertainty-driven offline reinforcement learning

C Bai, L Wang, Z Yang, Z Deng, A Garg, P Liu… - arxiv preprint arxiv …, 2022 - arxiv.org
Offline Reinforcement Learning (RL) aims to learn policies from previously collected
datasets without exploring the environment. Directly applying off-policy algorithms to offline …

Hybrid rl: Using both offline and online data can make rl efficient

Y Song, Y Zhou, A Sekhari, JA Bagnell… - arxiv preprint arxiv …, 2022 - arxiv.org
We consider a hybrid reinforcement learning setting (Hybrid RL), in which an agent has
access to an offline dataset and the ability to collect experience via real-world online …

Chipformer: Transferable chip placement via offline decision transformer

Y Lai, J Liu, Z Tang, B Wang, J Hao… - … on Machine Learning, 2023 - proceedings.mlr.press
Placement is a critical step in modern chip design, aiming to determine the positions of
circuit modules on the chip canvas. Recent works have shown that reinforcement learning …

Pre-training for robots: Offline rl enables learning new tasks from a handful of trials

A Kumar, A Singh, F Ebert, M Nakamoto… - arxiv preprint arxiv …, 2022 - arxiv.org
Progress in deep learning highlights the tremendous potential of utilizing diverse robotic
datasets for attaining effective generalization and makes it enticing to consider leveraging …