Shielding in Resource-Constrained Goal POMDPs

Investor logo

Warning

This publication doesn't include Faculty of Sports Studies. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

AJDARÓW Michal BRLEJ Šimon NOVOTNÝ Petr

Year of publication 2023
Type Article in Proceedings
Conference Proceedings of the 37th AAAI Conference on Artificial Intelligence
MU Faculty or unit

Faculty of Informatics

Citation
Web https://ojs.aaai.org/index.php/AAAI/article/view/26715
Doi http://dx.doi.org/10.1609/aaai.v37i12.26715
Keywords decision making; Markov decision processes; controller synthesis; resource constraints; shielding
Description We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info