Stop Gambling! It Just Takes Too Long

Research output: Contribution to book/Conference proceedings/Anthology/ReportChapter in book/Anthology/ReportContributedpeer-review

Abstract

The classical stochastic shortest path problem (SSPP) on integer-weighted Markov decision processes asks for a scheduler that maximizes the expected accumulated weight before reaching a goal state while ensuring that the goal is reached almost surely. In this article, we introduce the positively almost-surely terminating stochastic shortest path problem (PAST-SSPP), a variant of the classical SSPP, where the additional requirement is made that the expected number of steps for reaching the goal has to be finite. We show that PAST-SSPP can be solved in polynomial time. To this end, we provide an extension of the well-known spider construction as introduced by Christel Baier et al. used for the solution of the classical SSPP.

Details

Original languageEnglish
Title of host publicationPrinciples of Formal Quantitative Analysis
PublisherSpringer Science and Business Media B.V.
Pages139-157
Number of pages19
ISBN (electronic)978-3-031-97439-7
ISBN (print)978-3-031-97438-0
Publication statusE-pub ahead of print - 30 Aug 2025
Peer-reviewedYes

Publication series

SeriesLecture notes in computer science
VolumeLNCS 15760
ISSN0302-9743

External IDs

ORCID /0000-0003-4829-0476/work/194824700
ORCID /0000-0003-1724-2586/work/194825125

Keywords

Keywords

  • almost-sure termination, Markov decision processes, optional stopping theorem, Stochastic shortest path problem