Stochastic Games for Interactive Manipulation Domains

TL;DR

Stochastic games are proposed as a unifying framework for robot manipulation in the presence of strategic humans and stochastic action outcomes. The authors formalize a probabilistic abstraction of the manipulation domain, express tasks with $LTL_f$, and reduce strategy synthesis to a two-player turn-based stochastic game solvable with $PRISM ext{-}games$. They contribute a scalable model-construction approach, relax severe human-intervention assumptions, and release an open-source tool, plus demonstrations on case studies including a trembling-hand tic-tac-toe. The work enables robust, correct-by-construction robot strategies in interactive manipulation scenarios and points to future work on symbolic scaling and more complex agent behaviors.

Abstract

As robots become more prevalent, the complexity of robot-robot, robot-human, and robot-environment interactions increases. In these interactions, a robot needs to consider not only the effects of its own actions, but also the effects of other agents' actions and the possible interactions between agents. Previous works have considered reactive synthesis, where the human/environment is modeled as a deterministic, adversarial agent; as well as probabilistic synthesis, where the human/environment is modeled via a Markov chain. While they provide strong theoretical frameworks, there are still many aspects of human-robot interaction that cannot be fully expressed and many assumptions that must be made in each model. In this work, we propose stochastic games as a general model for human-robot interaction, which subsumes the expressivity of all previous representations. In addition, it allows us to make fewer modeling assumptions and leads to more natural and powerful models of interaction. We introduce the semantics of this abstraction and show how existing tools can be utilized to synthesize strategies to achieve complex tasks with guarantees. Further, we discuss the current computational limitations and improve the scalability by two orders of magnitude by a new way of constructing models for PRISM-games.

Figures (6)

• Figure 1: Tic-tac-toe game between a robot and a human. The robot is unaware of the level of expertise of the human and suffers from the "trembling hand" problem. In this case, the robot needs to reason about the probabilities of reaching a given state as well as the strategic responses of both agents from that state.
• Figure 2: Manipulation domain: (left) the locations of interest, where the Else location ($L_1$) contains all objects not otherwise shown. (right) Initial state with red and yellow blocks at $L_2$ and $L_3$ and the blue block at $L_1$.
• Figure 3: Example abstraction of manipulation domain from \ref{['fig:toy_example']} with stochasticity for robot actions.
• Figure 4: Stochastic game variant of MDP in \ref{['fig:example_mdp_robot']}. The circle and rectangle states belong to the robot and human player. For this e.g. we allow human to move objects from the robot's gripper. The top row shows multiple human movements, while the state on the right corresponds to no human intervention.
• Figure 5: Benchmark results for different scenarios using our approach. (a) and (d) illustrate the model construction time using the original PRISM-games and our implementation for the probabilistic human termination scenario. (b) and (e) illustrate computation times for the 1:1 action ratio scenario, and (c) and (f) correspond to the probabilistic human termination scenario.

Theorems & Definitions (7)

• Definition 1: Probabilistic Manipulation Domain Abstraction
• Example 1: MDP
• Definition 2: $\textsc{ltl}_f$ Syntax
• Example 2: Example $\textsc{ltl}_f$ specification
• Definition 3: Stochastic Game
• Example 3
• Remark 1