Value Iteration for Simple Stochastic Games: Stopping Criterion and Learning Algorithm

Investor logo

Warning

This publication doesn't include Faculty of Sports Studies. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

KELMENDI Edon KRÄMER Julia KŘETÍNSKÝ Jan WEININGER Maximilian

Year of publication 2018
Type Article in Proceedings
Conference Computer Aided Verification (CAV 2018)
MU Faculty or unit

Faculty of Informatics

Citation
Doi http://dx.doi.org/10.1007/978-3-319-96145-3_36
Keywords Value Iteration; Simple Stochastic Games; Stopping Criterion; Learning
Description Simple stochastic games can be solved by value iteration (VI), which yields a sequence of under-approximations of the value of the game. This sequence is guaranteed to converge to the value only in the limit. Since no stopping criterion is known, this technique does not provide any guarantees on its results. We provide the first stopping criterion for VI on simple stochastic games. It is achieved by additionally computing a convergent sequence of over-approximations of the value, relying on an analysis of the game graph. Consequently, VI becomes an anytime algorithm returning the approximation of the value and the current error bound. As another consequence, we can provide a simulation-based asynchronous VI algorithm, which yields the same guarantees, but without necessarily exploring the whole game graph.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info