This paper surveys various results about Markov chains on general (non-countable) state
spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms …

Abstract Markov chain Monte Carlo (MCMC) algorithms are generally regarded as the gold
standard technique for Bayesian inference. They are theoretically well-understood and …

A high school student can create deep Q-learning code to control her robot, without any
understanding of the meaning of'deep'or'Q', or why the code sometimes fails. This book is …

Heuristically, a discrete-time stochastic process has the Markov property if the past and
future are independent given the present. In this introductory chapter, we give the formal …

Sampling from various kinds of distribution is an issue of paramount importance in statistics
since it is often the key ingredient for constructing estimators, test procedures or confidence …

We study the problem of sampling from a strongly log-concave density supported on
$\mathbb {R}^ d $, and prove a non-asymptotic upper bound on the mixing time of the …

The ability to adapt to stimuli is a defining feature of many biological systems and critical to
maintaining homeostasis. While it is well appreciated that negative feedback can be used to …

Markov Chains and Stochastic Stability is part of the Communications and Control
Engineering Series (CCES) edited by Professors BW Dickinson, ED Sontag, M. Thoma, A …

This book provides a self-contained and up-to-date treatment of the Monte Carlo method
and develops a common framework under which various Monte Carlo techniques can be" …

While there have been few theoretical contributions on the Markov Chain Monte Carlo
(MCMC) methods in the past decade, current understanding and application of MCMC to the …