Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition

Jinghui Lu, Ziwei Yang, Yanjie Wang, Xuejing Liu, Brian Mac Namee, Can Huang

TL;DR

PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency and maintaining the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.

Abstract

In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.

PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition

TL;DR

PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency and maintaining the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.

Abstract

In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.
Paper Structure (35 sections, 1 equation, 4 figures, 18 tables)

This paper contains 35 sections, 1 equation, 4 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Models for NER
  4. Inference Speedup in LLMs
  5. Method
  6. Reframing of Instruction Tuning
  7. Inference of Label-Mention Pairs
  8. Removal of Duplicate Mentions
  9. Experiments
  10. Setup
  11. Datasets
  12. Training setup
  13. Inference setup
  14. Baselines
  15. Evaluation
  16. ...and 20 more sections

Figures (4)

  • Figure 1: PaDeLLM-NER training paradigm: texts within frames of the same color represents one training example, where texts inside the solid-line frame are the input, and those inside the dashed-line frame are the output. Italic texts are prompt templates. The "entity type" signifies the label being predicted. The "<num>" indicates count of mentions for that label, and "<mention n>" refers to the $n^{th}$ mention of a label in the input.
  • Figure 2: PaDeLLM-NER inference paradigm: texts enclosed in frames with identical colors indicate sequences of the same label. Specifically, the texts within solid-lined frames represent the added templates, while those within dashed-lined frames denote the prediction. In Step 1, the model predicts the number of mentions for all labels while in Step 2, it predicts the mentions. By aggregating mentions and labels from all sequences, the final NER results are obtained. Duplicate mentions appearing in different labels are resolved using prediction probabilities.
  • Figure 3: Speedup of PaDeLLM-NER compared to Autoregressive methods.
  • Figure 4: Percentage of different error types.