Compositional Value Iteration with Pareto Caching
Kazuki Watanabe, Marck van der Vegt, Sebastian Junges, Ichiro Hasuo
TL;DR
This work tackles the scalability of probabilistic model checking for large, compositionally structured MDPs by introducing Compositional Value Iteration (CVI) that operates on string diagrams of open MDPs. CVI runs value iteration on components in a top-down, by-need fashion and leverages Pareto caching to reuse results across repeated components, enabling more efficient verification of weighted reachability objectives. The authors propose two sound global stopping criteria: an optimistic value iteration (OVI) based criterion adapted to CVI and a bottom-up criterion that leverages Pareto caches, demonstrating favorable performance on benchmarks with trade-offs and overheads. The study highlights significant speedups in many cases while outlining challenges for termination guarantees and cache management, pointing to directions for asynchronous VI and further optimization.
Abstract
The de-facto standard approach in MDP verification is based on value iteration (VI). We propose compositional VI, a framework for model checking compositional MDPs, that addresses efficiency while maintaining soundness. Concretely, compositional MDPs naturally arise from the combination of individual components, and their structure can be expressed using, e.g., string diagrams. Towards efficiency, we observe that compositional VI repeatedly verifies individual components. We propose a technique called Pareto caching that allows to reuse verification results, even for previously unseen queries. Towards soundness, we present two stopping criteria: one generalizes the optimistic value iteration paradigm and the other uses Pareto caches in conjunction with recent baseline algorithms. Our experimental evaluations shows the promise of the novel algorithm and its variations, and identifies challenges for future work.