Distributionally robust model-based reinforcement learning with large state spaces

SS Ramesh, PG Sessa, Y Hu… - International …, 2024 - proceedings.mlr.press
Three major challenges in reinforcement learning are the complex dynamical systems with
large state spaces, the costly data acquisition processes, and the deviation of real-world …

Distributionally robust reinforcement learning with interactive data collection: Fundamental hardness and near-optimal algorithm

M Lu, H Zhong, T Zhang, J Blanchet - arxiv preprint arxiv:2404.03578, 2024 - arxiv.org
The sim-to-real gap, which represents the disparity between training and testing
environments, poses a significant challenge in reinforcement learning (RL). A promising …

[PDF][PDF] DRAGO: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization

R Mehta, J Diakonikolas… - The Thirty-eighth Annual …, 2024 - proceedings.neurips.cc
We consider the penalized distributionally robust optimization (DRO) problem with a closed,
convex uncertainty set, a setting that encompasses learning using f-DRO and spectral/L-risk …

A Primal-Dual Algorithm for Faster Distributionally Robust Optimization

R Mehta, J Diakonikolas, Z Harchaoui - arxiv preprint arxiv:2403.10763, 2024 - arxiv.org
We consider the penalized distributionally robust optimization (DRO) problem with a closed,
convex uncertainty set, a setting that encompasses the $ f $-DRO, Wasserstein-DRO, and …

Learning a Single Neuron Robustly to Distributional Shifts and Adversarial Label Noise

S Li, S Karmalkar, I Diakonikolas… - arxiv preprint arxiv …, 2024 - arxiv.org
We study the problem of learning a single neuron with respect to the $ L_2^ 2$-loss in the
presence of adversarial distribution shifts, where the labels can be arbitrary, and the goal is …

Relative-Translation Invariant Wasserstein Distance

B Wang, Q Di, M Yin, M Wang, Q Gu, P Wei - arxiv preprint arxiv …, 2024 - arxiv.org
We introduce a new family of distances, relative-translation invariant Wasserstein distances
($ RW_p $), for measuring the similarity of two probability distributions under distribution …

Policy Gradient for Robust Markov Decision Processes

Q Wang, S Xu, CP Ho, M Petrik - arxiv preprint arxiv:2410.22114, 2024 - arxiv.org
We develop a generic policy gradient method with the global optimality guarantee for robust
Markov Decision Processes (MDPs). While policy gradient methods are widely used for …

Wasserstein Distributionally Robust Control and State Estimation for Partially Observable Linear Systems

M Jang, A Hakobyan, I Yang - arxiv preprint arxiv:2406.01723, 2024 - arxiv.org
This paper presents a novel Wasserstein distributionally robust control and state estimation
algorithm for partially observable linear stochastic systems, where the probability …

Distributionally Robust Safety Verification for Markov Decision Processes

A Mazumdar, Y Hou, R Wisniewski - arxiv preprint arxiv:2411.15622, 2024 - arxiv.org
In this paper, we propose a distributionally robust safety verification method for Markov
decision processes where only an ambiguous transition kernel is available instead of the …

Dynamic Programs on Partially Ordered Sets

TJ Sargent, J Stachurski - arxiv preprint arxiv:2308.02148, 2023 - arxiv.org
We introduce a framework that represents a dynamic program as a family of operators acting
on a partially ordered set. We provide an optimality theory based only on order-theoretic …