The design and control of autonomous systems that operate in uncertain or adversarial
environments can be facilitated by formal modeling and analysis. Probabilistic model …

We present the probabilistic model checker Storm. Storm supports the analysis of discrete-
and continuous-time variants of both Markov chains and Markov decision processes. Storm …

This position paper reflects on the state-of-the-art in decision-making under uncertainty. A
classical assumption is that probabilities can sufficiently capture all uncertainty in a system …

This article focuses on inferring a general class of hidden Markov models (HMMs) using
data acquired from experts. Expert-acquired data contain decisions/actions made by …

We present a novel learning framework to obtain finite-state controllers (FSCs) for partially
observable Markov decision processes and illustrate its applicability for indefinite-horizon …

Markov models comprise states with probabilistic transitions. The analysis of these models is
ubiquitous and studied in, among others, reliability engineering, artificial intelligence …

Reinforcement Learning (RL) is a widely employed machine learning architecture that has
been applied to a variety of control problems. However, applications in safety-critical …

Uncertain partially observable Markov decision processes (uPOMDPs) allow the
probabilistic transition and observation functions of standard POMDPs to belong to a so …

This paper surveys the analysis of parametric Markov models whose transitions are labelled
with functions over a finite set of parameters. These models are symbolic representations of …

Abstract Partially-Observable Markov Decision Processes (POMDPs) are a well-known
stochastic model for sequential decision making under limited information. We consider the …