Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification

Jiayi Pan, Glen Chou, Dmitry Berenson

TL;DR

<3-5 sentence high-level summary> The paper tackles translating natural language commands into linear temporal logic (LTL) to specify robot tasks, aiming for high data efficiency. It introduces a data-synthesis pipeline combining back-translation and large-language-model paraphrasing along with constrained decoding to produce accurate NL-LTL mappings with very few human annotations. The approach yields competitive NL-to-LTL translation across three datasets, and, when combined with larger human-labeled data, improves generalization, enabling long-horizon planning on a 12D quadrotor. This work narrows the gap between NLP-based semantic parsing and formal task specification for robotics, reducing labeling costs while maintaining planning-quality translations.

Abstract

To make robots accessible to a broad audience, it is critical to endow them with the ability to take universal modes of communication, like commands given in natural language, and extract a concrete desired task specification, defined using a formal language like linear temporal logic (LTL). In this paper, we present a learning-based approach for translating from natural language commands to LTL specifications with very limited human-labeled training data. This is in stark contrast to existing natural-language to LTL translators, which require large human-labeled datasets, often in the form of labeled pairs of LTL formulas and natural language commands, to train the translator. To reduce reliance on human data, our approach generates a large synthetic training dataset through algorithmic generation of LTL formulas, conversion to structured English, and then exploiting the paraphrasing capabilities of modern large language models (LLMs) to synthesize a diverse corpus of natural language commands corresponding to the LTL formulas. We use this generated data to finetune an LLM and apply a constrained decoding procedure at inference time to ensure the returned LTL formula is syntactically correct. We evaluate our approach on three existing LTL/natural language datasets and show that we can translate natural language commands at 75\% accuracy with far less human data ($\le$12 annotations). Moreover, when training on large human-annotated datasets, our method achieves higher test accuracy (95\% on average) than prior work. Finally, we show the translated formulas can be used to plan long-horizon, multi-stage tasks on a 12D quadrotor.

Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification

TL;DR

<3-5 sentence high-level summary> The paper tackles translating natural language commands into linear temporal logic (LTL) to specify robot tasks, aiming for high data efficiency. It introduces a data-synthesis pipeline combining back-translation and large-language-model paraphrasing along with constrained decoding to produce accurate NL-LTL mappings with very few human annotations. The approach yields competitive NL-to-LTL translation across three datasets, and, when combined with larger human-labeled data, improves generalization, enabling long-horizon planning on a 12D quadrotor. This work narrows the gap between NLP-based semantic parsing and formal task specification for robotics, reducing labeling costs while maintaining planning-quality translations.

Abstract

To make robots accessible to a broad audience, it is critical to endow them with the ability to take universal modes of communication, like commands given in natural language, and extract a concrete desired task specification, defined using a formal language like linear temporal logic (LTL). In this paper, we present a learning-based approach for translating from natural language commands to LTL specifications with very limited human-labeled training data. This is in stark contrast to existing natural-language to LTL translators, which require large human-labeled datasets, often in the form of labeled pairs of LTL formulas and natural language commands, to train the translator. To reduce reliance on human data, our approach generates a large synthetic training dataset through algorithmic generation of LTL formulas, conversion to structured English, and then exploiting the paraphrasing capabilities of modern large language models (LLMs) to synthesize a diverse corpus of natural language commands corresponding to the LTL formulas. We use this generated data to finetune an LLM and apply a constrained decoding procedure at inference time to ensure the returned LTL formula is syntactically correct. We evaluate our approach on three existing LTL/natural language datasets and show that we can translate natural language commands at 75\% accuracy with far less human data (12 annotations). Moreover, when training on large human-annotated datasets, our method achieves higher test accuracy (95\% on average) than prior work. Finally, we show the translated formulas can be used to plan long-horizon, multi-stage tasks on a 12D quadrotor.
Paper Structure (30 sections, 2 equations, 4 figures, 1 table)

This paper contains 30 sections, 2 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Problem Statement
  4. Linear temporal logic (LTL)
  5. Generative Language Models
  6. Problem statement
  7. Method
  8. Data synthesis pipeline
  9. Back-translation
  10. Augmentation
  11. Architecture
  12. Pre-training and fine-tuning
  13. Canonical form for LTL
  14. Constrained decoding for the language model
  15. Results
  16. ...and 15 more sections

Figures (4)

  • Figure 1: We translate natural language commands into LTL formulas that achieve complex tasks on a 12D quadrotor.
  • Figure 2: Method flow: generating synthetic data, training on that data, evaluation, and planning with the evaluated formula.
  • Figure 3: Transforming from raw LTL to a canonical form.
  • Figure 4: The evaluation datasets. See Fig. \ref{['fig:demo']} for the drone dataset. (A) Cleanup World gopalan_sequence--sequence_2018. (B) Pick-and-place gopalan_sequence--sequence_2018.