Abstract Given a Markov Decision Process (MDP) with $ n $ states and $ m $ actions per
state, we study the number of iterations needed by Policy Iteration (PI) algorithms to …

This paper considers what we propose to call multi-gear bandits, which are Markov decision
processes modeling a generic dynamic and stochastic project fueled by a single resource …

We show subexponential lower bounds (ie, 2Ω (nc)) on the smoothed complexity of the
classical Howard's Policy Iteration algorithm for Markov Decision Processes. The bounds …

Recently discovered polyhedral structures of the value function for finite discounted Markov
decision processes (MDP) shed light on understanding the success of reinforcement …

Abstract Policy Iteration (PI)(Howard 1960) is a classical method for computing an optimal
policy for a finite Markov Decision Problem (MDP). The method is conceptually simple …

Abstract Unique Sink Orientations (USOs) are an appealing abstraction of several major
optimization problems of applied mathematics such as Linear Programming (LP), Markov …

Policy Iteration (PI) is a widely used family of algorithms to compute optimal policies for
Markov Decision Problems (MDPs). We derive upper bounds on the running time of PI on …

We introduce a framework to approximate Markov decision processes (MDPs) that stands on
two pillars:(i) state aggregation, as the algorithmic infrastructure, and (ii) central-limit …

Utilizing structure in mathematical modeling is instrumental for better model design, creation,
and solution. In this dissertation, we explore smoothness-based structure for problems …

Theoretical Analysis of Policy Iteration Page 1 1/31 Theoretical Analysis of Policy Iteration
Shivaram Kalyanakrishnan Department of Computer Science and Engineering Indian Institute …