RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU

TL;DR

RexUniNLU presents a unified encoder-based approach to universal NLU by redefining UIE and introducing an Explicit Schema Instructor to constrain extractions. It recursively queries all schema types with three token-linking operations, enabling extraction of complex schemas such as quadruples and quintuples across IE, CLS, and cross-modality tasks. The framework demonstrates strong performance in both high- and low-resource settings, and its cross-language and multi-modal variants show substantial robustness and competitive results. The work highlights the value of explicit schema constraints for accurate, scalable NLU and provides open-source resources to support broader adoption.

Abstract

Information Extraction (IE) and Text Classification (CLS) serve as the fundamental pillars of NLU, with both disciplines relying on analyzing input sequences to categorize outputs into pre-established schemas. However, there is no existing encoder-based model that can unify IE and CLS tasks from this perspective. To fully explore the foundation shared within NLU tasks, we have proposed a Recursive Method with Explicit Schema Instructor for Universal NLU. Specifically, we firstly redefine the true universal information extraction (UIE) with a formal formulation that covers almost all extraction schemas, including quadruples and quintuples which remain unsolved for previous UIE models. Then, we expands the formulation to all CLS and multi-modal NLU tasks. Based on that, we introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS, which encompasses all IE and CLS tasks and prevent incorrect connections between schema and input sequence. To avoid interference between different schemas, we reset the position ids and attention mask matrices. Extensive experiments are conducted on IE, CLS in both English and Chinese, and multi-modality, revealing the effectiveness and superiority. Our codes are publicly released.

Figures (6)

• Figure 1: Similarity between IE and CLS. IE focuses on establishing the relationship between pre-defined labels and specific components of the input sequence, whereas CLS aims to connect these labels with the entirety of the input sequence.
• Figure 2: Comparison of RexUniNLU with previous UIE. (a) The previous UIE models the information extraction task by defining the text spans and the relation between span pairs, but it is limited to extracting only two spans. (b) Our proposed RexUniNLU recursively extracts text spans for each type based on a given schema, and feeds the extracted information to the following extraction.
• Figure 3: The overall framework of RexUniNLU. We illustrate the computation process of the $i$-th query and the construction of the $i+1$-th query. $Y_i$ denotes the output of the $i$-th query, with all outputs ultimately combined to form the overall extraction result.
• Figure 4: (a) Queries and score matrices for NER and RE. The left sub-figure shows how to extract entities "Steve Jobs" and "Apple". The right sub-figure shows how to extract the relation given the entity "Steve Jobs" coupled with type "person". The schema is organized as {"person": {"work for (organization)": null}, "organization": null }. The score matrix is separated into three valid parts: token head-tail, type-token tail and token head-type. The cells scored as 1 are darken, the others are scored as 0. (b) The query and score matrix for text classification using RexUniNLU.
• Figure 5: Token type ids, position ids and Attention mask for RexUniNLU. $p$ and $t$ denote the prefix and types of the first group of previously extracted results. $q$ and $u$ denote the prefix and types for the second group.