The probabilistic model checker Storm

C Hensel, S Junges, JP Katoen, T Quatmann… - International Journal on …, 2022 - Springer
We present the probabilistic model checker Storm. Storm supports the analysis of discrete-
and continuous-time variants of both Markov chains and Markov decision processes. Storm …

Safe reinforcement learning via shielding under partial observability

S Carr, N Jansen, S Junges, U Topcu - Proceedings of the AAAI …, 2023 - ojs.aaai.org
Safe exploration is a common problem in reinforcement learning (RL) that aims to prevent
agents from making disastrous decisions while exploring their environment. A family of …

[PDF][PDF] Parameter synthesis in Markov models

S Junges - 2020 - publications.rwth-aachen.de
Markov models comprise states with probabilistic transitions. The analysis of these models is
ubiquitous and studied in, among others, reliability engineering, artificial intelligence …

Model-free, model-based, and general intelligence

H Geffner - arxiv preprint arxiv:1806.02308, 2018 - arxiv.org
During the 60s and 70s, AI researchers explored intuitions about intelligence by writing
programs that displayed intelligent behavior. Many good ideas came out from this work but …

[PDF][PDF] Finite-state controllers of POMDPs via parameter synthesis

JSL Junges, NH Jansen, R Wimmer, T Quatmann… - 2018 - repository.ubn.ru.nl
We study finite-state controllers (FSCs) for partially observable Markov decision processes
(POMDPs) that are provably correct with respect to given specifications. The key insight is …

Enforcing almost-sure reachability in POMDPs

S Junges, N Jansen, SA Seshia - International Conference on Computer …, 2021 - Springer
Abstract Partially-Observable Markov Decision Processes (POMDPs) are a well-known
stochastic model for sequential decision making under limited information. We consider the …

Verifiable RNN-based policies for POMDPs under temporal logic constraints

S Carr, N Jansen, U Topcu - arxiv preprint arxiv:2002.05615, 2020 - arxiv.org
Recurrent neural networks (RNNs) have emerged as an effective representation of control
policies in sequential decision-making problems. However, a major drawback in the …

Under-approximating expected total rewards in POMDPs

A Bork, JP Katoen, T Quatmann - … Conference on Tools and Algorithms for …, 2022 - Springer
We consider the problem: is the optimal expected total reward to reach a goal state in a
partially observable Markov decision process (POMDP) below a given threshold? We tackle …

Task-aware verifiable RNN-based policies for partially observable Markov decision processes

S Carr, N Jansen, U Topcu - Journal of Artificial Intelligence Research, 2021 - jair.org
Partially observable Markov decision processes (POMDPs) are models for sequential
decision-making under uncertainty and incomplete information. Machine learning methods …

Planning and SAT

J Rintanen - Handbook of Satisfiability, 2021 - ebooks.iospress.nl
The planning problem in Artificial Intelligence was the first application of SAT to reasoning
about transition systems and a direct precursor to the use of SAT in a number of other …