Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning specifications for reactive synthesis with safety constraints

Kandai Watanabe, Nicholas Renninger, Sriram Sankaranarayanan, Morteza Lahijanian

TL;DR

The paper tackles learning task specifications from demonstrations for robots operating in dynamic environments under safety constraints. It proposes learning a probabilistic deterministic finite automaton (PDFA) via safety-aware grammatical inference (EDSM) and synthesizing reactive strategies through a multi-objective product game, yielding a Pareto front over task preferences and robot costs. Safety is enforced either post-hoc or pre-emptively through a simulation relation, with the latter guaranteeing safety during learning. A polynomial-time Pareto-front computation and linear-time strategy extraction enable robust, explainable planning across static and dynamic settings, including manipulation tasks, validated by extensive case studies. Overall, the framework provides interpretable task specifications, safety assurances, and adaptable planning suitable for real-world robotic deployment.

Abstract

This paper presents a novel approach to learning from demonstration that enables robots to autonomously execute complex tasks in dynamic environments. We model latent tasks as probabilistic formal languages and introduce a tailored reactive synthesis framework that balances robot costs with user task preferences. Our methodology focuses on safety-constrained learning and inferring formal task specifications as Probabilistic Deterministic Finite Automata (PDFA). We adapt existing evidence-driven state merging algorithms and incorporate safety requirements throughout the learning process to ensure that the learned PDFA always complies with safety constraints. Furthermore, we introduce a multi-objective reactive synthesis algorithm that generates deterministic strategies that are guaranteed to satisfy the PDFA task while optimizing the trade-offs between user preferences and robot costs, resulting in a Pareto front of optimal solutions. Our approach models the interaction as a two-player game between the robot and the environment, accounting for dynamic changes. We present a computationally-tractable value iteration algorithm to generate the Pareto front and the corresponding deterministic strategies. Comprehensive experimental results demonstrate the effectiveness of our algorithms across various robots and tasks, showing that the learned PDFA never includes unsafe behaviors and that synthesized strategies consistently achieve the task while meeting both the robot cost and user-preference requirements.

Learning specifications for reactive synthesis with safety constraints

TL;DR

The paper tackles learning task specifications from demonstrations for robots operating in dynamic environments under safety constraints. It proposes learning a probabilistic deterministic finite automaton (PDFA) via safety-aware grammatical inference (EDSM) and synthesizing reactive strategies through a multi-objective product game, yielding a Pareto front over task preferences and robot costs. Safety is enforced either post-hoc or pre-emptively through a simulation relation, with the latter guaranteeing safety during learning. A polynomial-time Pareto-front computation and linear-time strategy extraction enable robust, explainable planning across static and dynamic settings, including manipulation tasks, validated by extensive case studies. Overall, the framework provides interpretable task specifications, safety assurances, and adaptable planning suitable for real-world robotic deployment.

Abstract

This paper presents a novel approach to learning from demonstration that enables robots to autonomously execute complex tasks in dynamic environments. We model latent tasks as probabilistic formal languages and introduce a tailored reactive synthesis framework that balances robot costs with user task preferences. Our methodology focuses on safety-constrained learning and inferring formal task specifications as Probabilistic Deterministic Finite Automata (PDFA). We adapt existing evidence-driven state merging algorithms and incorporate safety requirements throughout the learning process to ensure that the learned PDFA always complies with safety constraints. Furthermore, we introduce a multi-objective reactive synthesis algorithm that generates deterministic strategies that are guaranteed to satisfy the PDFA task while optimizing the trade-offs between user preferences and robot costs, resulting in a Pareto front of optimal solutions. Our approach models the interaction as a two-player game between the robot and the environment, accounting for dynamic changes. We present a computationally-tractable value iteration algorithm to generate the Pareto front and the corresponding deterministic strategies. Comprehensive experimental results demonstrate the effectiveness of our algorithms across various robots and tasks, showing that the learned PDFA never includes unsafe behaviors and that synthesized strategies consistently achieve the task while meeting both the robot cost and user-preference requirements.
Paper Structure (33 sections, 7 theorems, 12 equations, 20 figures, 1 table, 3 algorithms)

This paper contains 33 sections, 7 theorems, 12 equations, 20 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Two-player Game: Robot-Environment Interactive Model
  5. Task Specifications
  6. Safety Specifications
  7. Problem Formulation
  8. Safety Guaranteed PDFA Learning
  9. Grammatical Inference: PDFA Learning
  10. Learning with Safety Specification
  11. Post-process Algorithm
  12. Safety-Incorporated Learning Algorithm using "Pre-Processing"
  13. Reactive Strategy Synthesis with PDFA
  14. Product Construction
  15. Pareto Front Computation
  16. ...and 18 more sections

Key Result

Theorem 1

Let $R$ be a simulation relation between automata $\mathcal{A}$ and $\mathcal{B}$. It follows that $\mathcal{L}(\mathcal{A}) \subseteq \mathcal{L}(\mathcal{B})$.

Figures (20)

  • Figure 1: Schematic of an autonomous deep-sea science mission. The task for the green underwater vehicle is to visit a school of fish (blue) and shipwreck (brown) while avoiding coral reefs (yellow). Our goal is to infer the underlying task from the demonstrations (the colored path) and synthesize a controller in the trained/new environment that achieves the learned task.
  • Figure 2: DFA and PDFA representation of an autonomous deep-sea science mission.
  • Figure 3: Safety Violating DFA, Safety DFA, and Minimized Safety DFA
  • Figure 4: FPTA
  • Figure 5: Intermediate
  • ...and 15 more figures

Theorems & Definitions (29)

  • Definition 1: Two-player Game
  • Example 1: Two-player game
  • Definition 2: Play & Trace
  • Example 2: Play and Observation Trace
  • Definition 3: Multi-objective total payoff
  • Definition 4: Strategy
  • Definition 5: DFA
  • Example 3
  • Definition 6: PDFA
  • Example 4: PDFA
  • ...and 19 more