Bridging Perception and Planning: Towards End-to-End Planning for Signal Temporal Logic Tasks

TL;DR

S-MSP is proposed, a differentiable framework that maps synchronized multi-view camera observations and an STL specification directly to a feasible trajectory, and a rule-based safety filter at inference improves physical executability without compromising logical correctness, showcasing the practicality of the approach.

Abstract

We investigate the task and motion planning problem for Signal Temporal Logic (STL) specifications in robotics. Existing STL methods rely on pre-defined maps or mobility representations, which are ineffective in unstructured real-world environments. We propose the \emph{Structured-MoE STL Planner} (\textbf{S-MSP}), a differentiable framework that maps synchronized multi-view camera observations and an STL specification directly to a feasible trajectory. S-MSP integrates STL constraints within a unified pipeline, trained with a composite loss that combines trajectory reconstruction and STL robustness. A \emph{structure-aware} Mixture-of-Experts (MoE) model enables horizon-aware specialization by projecting sub-tasks into temporally anchored embeddings. We evaluate S-MSP using a high-fidelity simulation of factory-logistics scenarios with temporally constrained tasks. Experiments show that S-MSP outperforms single-expert baselines in STL satisfaction and trajectory feasibility. A rule-based \emph{safety filter} at inference improves physical executability without compromising logical correctness, showcasing the practicality of the approach.

Figures (2)

• Figure 1: The Overall Architecture of Model.
• Figure 2: Qualitative rollouts on representative tasks from ID and OOD suites.

Theorems & Definitions (1)

• Remark 1