Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models

Jun Xu, Mengshu Sun, Zhiqiang Zhang, Jun Zhou

TL;DR

The experimental results demonstrate that ChatUIE can significantly improve the performance of information extraction with a slight decrease in chatting ability and integrate generation constraints to address the issue of generating elements that are not present in the input.

Abstract

Recent advancements in large language models have shown impressive performance in general chat. However, their domain-specific capabilities, particularly in information extraction, have certain limitations. Extracting structured information from natural language that deviates from known schemas or instructions has proven challenging for previous prompt-based methods. This motivated us to explore domain-specific modeling in chat-based language models as a solution for extracting structured information from natural language. In this paper, we present ChatUIE, an innovative unified information extraction framework built upon ChatGLM. Simultaneously, reinforcement learning is employed to improve and align various tasks that involve confusing and limited samples. Furthermore, we integrate generation constraints to address the issue of generating elements that are not present in the input. Our experimental results demonstrate that ChatUIE can significantly improve the performance of information extraction with a slight decrease in chatting ability.

ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models

TL;DR

The experimental results demonstrate that ChatUIE can significantly improve the performance of information extraction with a slight decrease in chatting ability and integrate generation constraints to address the issue of generating elements that are not present in the input.

Abstract

Recent advancements in large language models have shown impressive performance in general chat. However, their domain-specific capabilities, particularly in information extraction, have certain limitations. Extracting structured information from natural language that deviates from known schemas or instructions has proven challenging for previous prompt-based methods. This motivated us to explore domain-specific modeling in chat-based language models as a solution for extracting structured information from natural language. In this paper, we present ChatUIE, an innovative unified information extraction framework built upon ChatGLM. Simultaneously, reinforcement learning is employed to improve and align various tasks that involve confusing and limited samples. Furthermore, we integrate generation constraints to address the issue of generating elements that are not present in the input. Our experimental results demonstrate that ChatUIE can significantly improve the performance of information extraction with a slight decrease in chatting ability.
Paper Structure (15 sections, 5 equations, 6 figures, 8 tables)

This paper contains 15 sections, 5 equations, 6 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Domain Knowledge Integration
  4. Reinforcement Learning
  5. Experimental Settings and Results
  6. Experimental Settings
  7. Results
  8. Overall Results
  9. Results For Chatting
  10. Zero-Shot Information Extraction
  11. Case Study
  12. Conclusion
  13. Limitations
  14. Acknowledgements
  15. Bibliographical References

Figures (6)

  • Figure 1: The approximate performance of unified information extraction framework in various application scenarios.
  • Figure 2: The overall framework of chat-based unified information extraction using LLMs.
  • Figure 3: Strategies for generating constraints in information extraction tasks.
  • Figure 4: Performance comparison of confusing and limited samples after reinforcement learning in FewFC.
  • Figure 5: Performance of different tasks as the number of training epochs increase.
  • ...and 1 more figures