Path-Constrained Markov Decision Processes: bridging the gap between probabilistic model-checking and decision-theoretic planning.

Abstract : Markov Decision Processes (MDPs) are a popular model for planning under probabilistic uncertainties. The solution of an MDP is a policy represented as a controlled Markov chain, whose complex properties on execution paths can be automatically validated using stochastic model-checking techniques. In this paper, we propose a new theoretical model, named Path-Constrained Markov Decision Processes: it allows system designers to directly optimize safe policies in a single design pass, whose possible executions are guaranteed to satisfy some probabilistic constraints on their paths, expressed in Probabilistic Real Time Computation Tree Logic. We mathematically analyze properties of PC-MDPs and provide an iterative linear programming algorithm for solving them.We also present experiments that illustrate PC-MDPs and highlight their benefits.
Type de document :
Communication dans un congrès
20th European Conference on Artificial Intelligence (ECAI 2012), Aug 2012, MONTPELLIER, France
Liste complète des métadonnées

Littérature citée [8 références]  Voir  Masquer  Télécharger

https://hal-onera.archives-ouvertes.fr/hal-01060349
Contributeur : Alain Broc <>
Soumis le : mercredi 3 septembre 2014 - 14:37:38
Dernière modification le : mercredi 28 mars 2018 - 14:16:10
Document(s) archivé(s) le : jeudi 4 décembre 2014 - 11:22:11

Fichier

DCSD12084.1399388482.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01060349, version 1

Collections

Citation

Florent Teichteil-Königsbuch. Path-Constrained Markov Decision Processes: bridging the gap between probabilistic model-checking and decision-theoretic planning.. 20th European Conference on Artificial Intelligence (ECAI 2012), Aug 2012, MONTPELLIER, France. 〈hal-01060349〉

Partager

Métriques

Consultations de la notice

299

Téléchargements de fichiers

250