Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems

Jialiang Fan, Weizhe Xu, Mengyu Liu, Oleg Sokolsky, Insup Lee, Fangxin Kong

TL;DR

SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains, and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats.

Abstract

Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).

SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems

TL;DR

SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains, and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats.

Abstract

Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).
Paper Structure (36 sections, 9 equations, 6 figures, 9 tables)

This paper contains 36 sections, 9 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classical Task Planning
  4. Learning-based Planners
  5. LLM-based Planning
  6. Preliminaries
  7. Classical Planning and PDDL
  8. Robotic Task Planning
  9. Safety-Aware Planning.
  10. Safety-Generalizable Planning.
  11. SafeGen-LLM
  12. Dataset Construction
  13. Supervised Fine-Tuning
  14. Group Relative Policy Optimization
  15. Reward design.
  16. ...and 21 more sections

Figures (6)

  • Figure 1: Overview of the proposed SafeGen-LLM framework. Stage I performs SFT on formally verified, safety-constrained plans. Stage II applies GRPO using fine-grained reward signals derived from formal verification to enforce safety alignment.
  • Figure 2: Pipeline for dataset construction.
  • Figure 3: Three-way benchmark comparison: GPT-5.2 vs OPTIC vs Fast Downward. Green squares = success, red circles = failure.
  • Figure 4: Error type distribution across training stages (Pretrained $\rightarrow$ SFT $\rightarrow$ GRPO) for Mistral-7B in the Blocksworld domain.
  • Figure 5: Cross-model success rate comparison across four domains.
  • ...and 1 more figures