We consider online reinforcement learning in Mean-Field Games (MFGs). Unlike traditional
approaches, we alleviate the need for a mean-field oracle by develo** an algorithm that …

We study stochastic approximation procedures for approximately solving a $ d $-
dimensional linear fixed point equation based on observing a trajectory of length $ n $ from …

We address the problem of estimating the mixing time $ t_ {\mathsf {mix}} $ of an arbitrary
ergodic finite Markov chain from a single trajectory of length $ m $. The reversible case was …

The focus of this paper is on stochastic variational inequalities (VI) under Markovian noise. A
prominent application of our algorithmic developments is the stochastic policy evaluation …

We study average-reward Markov decision processes (AMDPs) and develop novel first-
order methods with strong theoretical guarantees for both policy optimization and policy …

Policy optimization from batch data, ie, offline reinforcement learning (RL) is important when
collecting data from a current policy is not possible. This setting incurs distribution mismatch …

Information geometry and Markov chains are two powerful tools used in modern fields such
as finance, physics, computer science, and epidemiology. In this survey, we explore their …

We analyze the information geometric structure of time reversibility for parametric families of
irreducible transition kernels of Markov chains. We define and characterize reversible …

Importance Sampling (IS) is a method for approximating expectations with respect to a target
distribution using independent samples from a proposal distribution and the associated to …

We study the problem of estimating the stationary mass---also called the unigram mass---
that is missing from a single trajectory of a discrete-time, ergodic Markov chain. This problem …