Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hypernetworks for Perspectivist Adaptation

Daniil Ignatev, Denis Paperno, Massimo Poesio

TL;DR

The paper tackles the parametric bottleneck of perspectivist text classification by employing a hypernetwork to generate per-annotator adapter weights, keeping the base model frozen to avoid catastrophic forgetting. This architecture-agnostic approach uses low-rank LoRA adapters and embeds annotator and layer identifiers to produce $A_{ij}$ and $B_{ij}$ with $W_j' = W_j + B_j A_j$, enabling scalable personalization with only about $5.6\times 10^6$ trainable parameters. Empirical results on four perspectivist datasets show competitive performance against state-of-the-art annotator-aware methods, with notable gains on MDA and RB and improved item-level disagreement correlations on EPIC, while maintaining strong parameter efficiency. The work emphasizes the practical benefits of parameter-efficient perspectivist modeling, including reduced forgetting risk and easier scaling to many annotators, while acknowledging limitations in inference speed and handling unseen annotators. Overall, the hypernetwork+adapters strategy offers a promising, scalable direction for perspectivist learning in subjective NLP tasks.

Abstract

The task of perspective-aware classification introduces a bottleneck in terms of parametric efficiency that did not get enough recognition in existing studies. In this article, we aim to address this issue by applying an existing architecture, the hypernetwork+adapters combination, to perspectivist classification. Ultimately, we arrive at a solution that can compete with specialized models in adopting user perspectives on hate speech and toxicity detection, while also making use of considerably fewer parameters. Our solution is architecture-agnostic and can be applied to a wide range of base models out of the box.

Hypernetworks for Perspectivist Adaptation

TL;DR

The paper tackles the parametric bottleneck of perspectivist text classification by employing a hypernetwork to generate per-annotator adapter weights, keeping the base model frozen to avoid catastrophic forgetting. This architecture-agnostic approach uses low-rank LoRA adapters and embeds annotator and layer identifiers to produce and with , enabling scalable personalization with only about trainable parameters. Empirical results on four perspectivist datasets show competitive performance against state-of-the-art annotator-aware methods, with notable gains on MDA and RB and improved item-level disagreement correlations on EPIC, while maintaining strong parameter efficiency. The work emphasizes the practical benefits of parameter-efficient perspectivist modeling, including reduced forgetting risk and easier scaling to many annotators, while acknowledging limitations in inference speed and handling unseen annotators. Overall, the hypernetwork+adapters strategy offers a promising, scalable direction for perspectivist learning in subjective NLP tasks.

Abstract

The task of perspective-aware classification introduces a bottleneck in terms of parametric efficiency that did not get enough recognition in existing studies. In this article, we aim to address this issue by applying an existing architecture, the hypernetwork+adapters combination, to perspectivist classification. Ultimately, we arrive at a solution that can compete with specialized models in adopting user perspectives on hate speech and toxicity detection, while also making use of considerably fewer parameters. Our solution is architecture-agnostic and can be applied to a wide range of base models out of the box.
Paper Structure (27 sections, 5 equations, 1 figure, 5 tables)

This paper contains 27 sections, 5 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Data perspectivism:
  4. Perspectivist learning:
  5. Hypernetworks:
  6. Method
  7. Architecture
  8. Data
  9. The Multi-Domain Agreement dataset ($\mathcal{D}_{\textsl{MDA}}$):
  10. English Perspectivist Irony Corpus ($\mathcal{D}_{\textsl{EPIC}}$):
  11. The Racial Bias Toxicity Detection Corpus ($\mathcal{D}_{\textsl{RB}}$):
  12. Hate Speech Brexit ($\mathcal{D}_{\textsl{HS-Brexit}}$):
  13. Experiments
  14. Baselines
  15. AART:
  16. ...and 12 more sections

Figures (1)

  • Figure 1: Data flow within our hypernetwork implementation: the annotator id and the layer id are embedded to adapt the target layer $l_j$ in the base model.