Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sketch: A Toolkit for Streamlining LLM Operations

Xin Jiang, Xiang Li, Wenjia Ma, Xuezhi Fang, Yiqun Yao, Naitong Yu, Xuying Meng, Peng Han, Jing Li, Aixin Sun, Yequan Wang

TL;DR

Large language models excel at many tasks but struggle to produce strictly formatted outputs required by downstream systems. Sketch presents a schema-driven toolkit that standardizes task descriptions, task instantiation, prompt packaging, and controlled generation, paired with a fine-tuned Sketch-8B model built on LLaMA3-8B-Instruct. The authors show strong generalization to unseen formats, domains, and tasks through schema-following data and constrained decoding, achieving high formatting fidelity while maintaining task performance. The work enables plug-and-play deployment of structured-output LLM applications and provides open-source resources to the community.

Abstract

Large language models (LLMs) represented by GPT family have achieved remarkable success. The characteristics of LLMs lie in their ability to accommodate a wide range of tasks through a generative approach. However, the flexibility of their output format poses challenges in controlling and harnessing the model's outputs, thereby constraining the application of LLMs in various domains. In this work, we present Sketch, an innovative toolkit designed to streamline LLM operations across diverse fields. Sketch comprises the following components: (1) a suite of task description schemas and prompt templates encompassing various NLP tasks; (2) a user-friendly, interactive process for building structured output LLM services tailored to various NLP tasks; (3) an open-source dataset for output format control, along with tools for dataset construction; and (4) an open-source model based on LLaMA3-8B-Instruct that adeptly comprehends and adheres to output formatting instructions. We anticipate this initiative to bring considerable convenience to LLM users, achieving the goal of ''plug-and-play'' for various applications. The components of Sketch will be progressively open-sourced at https://github.com/cofe-ai/Sketch.

Sketch: A Toolkit for Streamlining LLM Operations

TL;DR

Large language models excel at many tasks but struggle to produce strictly formatted outputs required by downstream systems. Sketch presents a schema-driven toolkit that standardizes task descriptions, task instantiation, prompt packaging, and controlled generation, paired with a fine-tuned Sketch-8B model built on LLaMA3-8B-Instruct. The authors show strong generalization to unseen formats, domains, and tasks through schema-following data and constrained decoding, achieving high formatting fidelity while maintaining task performance. The work enables plug-and-play deployment of structured-output LLM applications and provides open-source resources to the community.

Abstract

Large language models (LLMs) represented by GPT family have achieved remarkable success. The characteristics of LLMs lie in their ability to accommodate a wide range of tasks through a generative approach. However, the flexibility of their output format poses challenges in controlling and harnessing the model's outputs, thereby constraining the application of LLMs in various domains. In this work, we present Sketch, an innovative toolkit designed to streamline LLM operations across diverse fields. Sketch comprises the following components: (1) a suite of task description schemas and prompt templates encompassing various NLP tasks; (2) a user-friendly, interactive process for building structured output LLM services tailored to various NLP tasks; (3) an open-source dataset for output format control, along with tools for dataset construction; and (4) an open-source model based on LLaMA3-8B-Instruct that adeptly comprehends and adheres to output formatting instructions. We anticipate this initiative to bring considerable convenience to LLM users, achieving the goal of ''plug-and-play'' for various applications. The components of Sketch will be progressively open-sourced at https://github.com/cofe-ai/Sketch.
Paper Structure (29 sections, 1 equation, 1 figure, 10 tables)

This paper contains 29 sections, 1 equation, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. Sketch Architecture
  3. Schema Selection
  4. Task Instantiation
  5. Prompt Packaging
  6. Generation
  7. Code Example
  8. Sketch-8B Fine-tuning Approach
  9. Data Preparation
  10. Fine-tuning Method
  11. Experiments
  12. Experiment Settings
  13. Experiment Data Settings.
  14. Fine-tuning Settings.
  15. Evaluation Methods.
  16. ...and 14 more sections

Figures (1)

  • Figure 1: Sketch workflow, taking a NER task CoNLL-2003 as an example. The nodes (Format Control and Format Validation) in slight yellow are optional.