In the last few years, reinforcement learning (RL), also called adaptive (or approximate)
dynamic programming, has emerged as a powerful tool for solving complex sequential …

A high school student can create deep Q-learning code to control her robot, without any
understanding of the meaning of'deep'or'Q', or why the code sometimes fails. This book is …

Heuristically, a discrete-time stochastic process has the Markov property if the past and
future are independent given the present. In this introductory chapter, we give the formal …

Abstract Throughout its history, Operational Research has evolved to include methods,
models and algorithms that have been applied to a wide range of contexts. This …

Markov Chains and Stochastic Stability is part of the Communications and Control
Engineering Series (CCES) edited by Professors BW Dickinson, ED Sontag, M. Thoma, A …

It is shown here that stability of the stochastic approximation algorithm is implied by the
asymptotic stability of the origin for an associated ODE. This in turn implies convergence of …

It is now known that the usual traffic condition (the nominal load being less than 1 at each
station) is not sufficient for stability for a multiclass open queueing network. Although there …

Our goal in writing this book has been to provide a comprehensive, mathematically rigorous,
but still accessible treatment of the interaction between information and control in multi …

Power grids, flexible manufacturing, cellular communications: interconnectedness has
consequences. This remarkable book gives the tools and philosophy you need to build …

We consider the following queueing system which arises as a model of a wireless link
shared by multiple users. Multiple ows must be served by a\channel"(server). The channel …