Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPT3-to-plan: Extracting plans from text using GPT-3

Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati

TL;DR

<3-5 sentence high-level summary>

Abstract

Operations in many essential industries including finance and banking are often characterized by the need to perform repetitive sequential tasks. Despite their criticality to the business, workflows are rarely fully automated or even formally specified, though there may exist a number of natural language documents describing these procedures for the employees of the company. Plan extraction methods provide us with the possibility of extracting structure plans from such natural language descriptions of the plans/workflows, which could then be leveraged by an automated system. In this paper, we investigate the utility of generalized language models in performing such extractions directly from such texts. Such models have already been shown to be quite effective in multiple translation tasks, and our initial results seem to point to their effectiveness also in the context of plan extractions. Particularly, we show that GPT-3 is able to generate plan extraction results that are comparable to many of the current state of the art plan extraction methods.

GPT3-to-plan: Extracting plans from text using GPT-3

TL;DR

<3-5 sentence high-level summary>

Abstract

Operations in many essential industries including finance and banking are often characterized by the need to perform repetitive sequential tasks. Despite their criticality to the business, workflows are rarely fully automated or even formally specified, though there may exist a number of natural language documents describing these procedures for the employees of the company. Plan extraction methods provide us with the possibility of extracting structure plans from such natural language descriptions of the plans/workflows, which could then be leveraged by an automated system. In this paper, we investigate the utility of generalized language models in performing such extractions directly from such texts. Such models have already been shown to be quite effective in multiple translation tasks, and our initial results seem to point to their effectiveness also in the context of plan extractions. Particularly, we show that GPT-3 is able to generate plan extraction results that are comparable to many of the current state of the art plan extraction methods.

Paper Structure

This paper contains 11 sections, 1 equation, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Experiments
  4. Datasets and GPT-3 API
  5. Query generation
  6. Evaluation and Metrics
  7. Baselines
  8. Results
  9. Ordering
  10. Discussion and Conclusion
  11. Acknowledgements

Figures (2)

  • Figure 1: $F_{1}$ scores of the model on the Windows Help and Support dataset for 1 to 4 few-shot training
  • Figure 2: Query examples on WHS, CT and WHG. Each query was input to Davinci along with two preceding training instances containing the largest proportion of optional and exclusive actions. The output is shown in regular text while the input is displayed in bold.