The variance-penalized stochastic shortest path problem

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Abstract

The stochastic shortest path problem (SSPP) asks to resolve the non-deterministic choices in a Markov decision process (MDP) such that the expected accumulated weight before reaching a target state is maximized. This paper addresses the optimization of the variance-penalized expectation (VPE) of the accumulated weight, which is a variant of the SSPP in which a multiple of the variance of accumulated weights is incurred as a penalty. It is shown that the optimal VPE in MDPs with non-negative weights as well as an optimal deterministic finite-memory scheduler can be computed in exponential space. The threshold problem whether the maximal VPE exceeds a given rational is shown to be EXPTIME-hard and to lie in NEXPTIME. Furthermore, a result of interest in its own right obtained on the way is that a variance-minimal scheduler among all expectation-optimal schedulers can be computed in polynomial time.

Details

Original languageEnglish
Title of host publication49th EATCS International Conference on Automata, Languages, and Programming, ICALP 2022
EditorsMikołaj Bojańczyk, Emanuela Merelli, David P. Woodruff
PublisherSchloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing
Pages129:1–129:19
Number of pages19
ISBN (electronic)9783959772358
ISBN (print)978-3-95977-235-8
Publication statusPublished - 1 Jul 2022
Peer-reviewedYes

Publication series

Series49th EATCS International Conference on Automata, Languages, and Programming (ICALP 2022) ; Vol. 229
Volume229
ISSN1868-8969

Conference

Title49th EATCS International Colloquium on Automata, Languages and Programming
Abbreviated titleICALP 2022
Duration4 - 8 July 2022
Degree of recognitionInternational event
Locationhybrid
CityParis
CountryFrance

External IDs

Scopus 85133468318
dblp conf/icalp/PiribauerSB22
Mendeley 4b556c64-0cd5-3b56-a6aa-bf19263a8017
ORCID /0000-0002-5321-9343/work/142236696
ORCID /0000-0003-4829-0476/work/165453933

Keywords

Research priority areas of TU Dresden

DFG Classification of Subject Areas according to Review Boards

Subject groups, research areas, subject areas according to Destatis

ASJC Scopus subject areas

Keywords

  • Markov decision process, variance, stochastic shortest path problem