In this review/tutorial article, we present recent progress on optimal control of partially
observed Markov Decision Processes (POMDPs). We first present regularity and continuity …

In this paper, for partially observed Markov decision problems (POMDPs), we provide the
convergence of a Q learning algorithm for control policies using a finite history of past …

As a primary contribution, we present a convergence theorem for stochastic iterations, and in
particular, Q-learning iterates, under a general, possibly non-Markovian, stochastic …

Reinforcement learning algorithms often require finiteness of state and action spaces in
Markov decision processes (MDPs)(also called controlled Markov chains) and various …

In the theory of Partially Observed Markov Decision Processes (POMDPs), existence of
optimal policies have in general been established via converting the original partially …

The study of continuous-time Markov decision processes dates back at least to the 1950s,
shortly after that of its discrete-time analogue. Since then, the theory has rapidly developed …

Control and optimization of dynamical systems in the presence of stochastic uncertainty is a
mature field with a large range of applications. A comprehensive treatment of such problems …

Establishing the existence of Nash equilibria for partially observed stochastic dynamic
games is known to be quite challenging, with the difficulties stemming from the noisy nature …

For sequential stochastic control problems with standard Borel measurement and control
action spaces, we introduce a general (universally applicable) dynamic programming …

The average cost optimality is known to be a challenging problem for partially observable
stochastic control, with few results available beyond the finite state, action, and …