Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Meta-Learning for Zero-Shot Relation Triplet Extraction

Wanli Li, Tieyun Qian, Yi Song, Zeyu Zhang, Jiawei Li, Zhuang Chen, Lixin Zou

TL;DR

Zero-shot Relation Triplet Extraction (ZeroRTE) requires identifying unseen relation types from text. The authors propose a Generative Meta-Learning framework built on bi-level optimization (BLO) that separates data fitting (LL) from generalization (UL) and introduces a task-aware generative model (TGM). They instantiate three BLO-driven meta-learning variants—TGM-Metric, TGM-Model, and TGM-Optimization—to capture cross-task knowledge directly from training data, without relying on synthesized data alone. Empirical results on FewRel and Wiki-ZSL demonstrate superior generalization over strong baselines and competitive performance against large language models, with detailed analyses of prompt settings and hyper-parameters. The work provides a BLO-based pathway to robust ZeroRTE, enabling more reliable extraction of unseen relations in practical IR and NLP tasks.

Abstract

Zero-shot Relation Triplet Extraction (ZeroRTE) aims to extract relation triplets from texts containing unseen relation types. This capability benefits various downstream information retrieval (IR) tasks. The primary challenge lies in enabling models to generalize effectively to unseen relation categories. Existing approaches typically leverage the knowledge embedded in pre-trained language models to accomplish the generalization process. However, these methods focus solely on fitting the training data during training, without specifically improving the model's generalization performance, resulting in limited generalization capability. For this reason, we explore the integration of bi-level optimization (BLO) with pre-trained language models for learning generalized knowledge directly from the training data, and propose a generative meta-learning framework which exploits the `learning-to-learn' ability of meta-learning to boost the generalization capability of generative models. Specifically, we introduce a BLO approach that simultaneously addresses data fitting and generalization. This is achieved by constructing an upper-level loss to focus on generalization and a lower-level loss to ensure accurate data fitting. Building on this, we subsequently develop three generative meta-learning methods, each tailored to a distinct category of meta-learning. Extensive experimental results demonstrate that our framework performs well on the ZeroRTE task. Our code is available at https://github.com/leeworry/TGM-MetaLearning.

Generative Meta-Learning for Zero-Shot Relation Triplet Extraction

TL;DR

Zero-shot Relation Triplet Extraction (ZeroRTE) requires identifying unseen relation types from text. The authors propose a Generative Meta-Learning framework built on bi-level optimization (BLO) that separates data fitting (LL) from generalization (UL) and introduces a task-aware generative model (TGM). They instantiate three BLO-driven meta-learning variants—TGM-Metric, TGM-Model, and TGM-Optimization—to capture cross-task knowledge directly from training data, without relying on synthesized data alone. Empirical results on FewRel and Wiki-ZSL demonstrate superior generalization over strong baselines and competitive performance against large language models, with detailed analyses of prompt settings and hyper-parameters. The work provides a BLO-based pathway to robust ZeroRTE, enabling more reliable extraction of unseen relations in practical IR and NLP tasks.

Abstract

Zero-shot Relation Triplet Extraction (ZeroRTE) aims to extract relation triplets from texts containing unseen relation types. This capability benefits various downstream information retrieval (IR) tasks. The primary challenge lies in enabling models to generalize effectively to unseen relation categories. Existing approaches typically leverage the knowledge embedded in pre-trained language models to accomplish the generalization process. However, these methods focus solely on fitting the training data during training, without specifically improving the model's generalization performance, resulting in limited generalization capability. For this reason, we explore the integration of bi-level optimization (BLO) with pre-trained language models for learning generalized knowledge directly from the training data, and propose a generative meta-learning framework which exploits the `learning-to-learn' ability of meta-learning to boost the generalization capability of generative models. Specifically, we introduce a BLO approach that simultaneously addresses data fitting and generalization. This is achieved by constructing an upper-level loss to focus on generalization and a lower-level loss to ensure accurate data fitting. Building on this, we subsequently develop three generative meta-learning methods, each tailored to a distinct category of meta-learning. Extensive experimental results demonstrate that our framework performs well on the ZeroRTE task. Our code is available at https://github.com/leeworry/TGM-MetaLearning.
Paper Structure (26 sections, 17 equations, 4 figures, 6 tables)

This paper contains 26 sections, 17 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Zero-shot Relation Triplet Extraction
  4. Bi-level Optimization
  5. Meta-learning
  6. Methodology
  7. Problem Formulation
  8. Framework Analysis
  9. Model Overview
  10. Task-aware Generative Model
  11. Metric-based Generative Meta-learning (TGM-Metric)
  12. Model-based Generative Meta-learning (TGM-Model)
  13. Optimization-based Generative Meta-learning (TGM-Optimization)
  14. Inference
  15. Experiments
  16. ...and 11 more sections

Figures (4)

  • Figure 1: An illustration of the RTE task and the difference between existing methods for ZeroRTE.
  • Figure 2: An overview of our proposed generative meta-learning framework for zero-shot RTE.
  • Figure 3: Case study. The orange, blue, and green tokens respectively denote the head entity, tail entity, and relation. Incorrectly extracted tokens are marked in grey.
  • Figure 4: Impacts of the hyper-parameters.