Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantically Orthogonal Framework for Citation Classification: Disentangling Intent and Content

Changxu Duan, Zhiyin Tan

TL;DR

Evaluation with both zero-shot and fine-tuned Large Language Models demonstrates that SOFT enables higher agreement between human annotators and LLMs, and supports stronger classification performance and robust cross-domain generalization compared to ACL-ARC and SciCite annotation frameworks.

Abstract

Understanding the role of citations is essential for research assessment and citation-aware digital libraries. However, existing citation classification frameworks often conflate citation intent (why a work is cited) with cited content type (what part is cited), limiting their effectiveness in auto classification due to a dilemma between fine-grained type distinctions and practical classification reliability. We introduce SOFT, a Semantically Orthogonal Framework with Two dimensions that explicitly separates citation intent from cited content type, drawing inspiration from semantic role theory. We systematically re-annotate the ACL-ARC dataset using SOFT and release a cross-disciplinary test set sampled from ACT2. Evaluation with both zero-shot and fine-tuned Large Language Models demonstrates that SOFT enables higher agreement between human annotators and LLMs, and supports stronger classification performance and robust cross-domain generalization compared to ACL-ARC and SciCite annotation frameworks. These results confirm SOFT's value as a clear, reusable annotation standard, improving clarity, consistency, and generalizability for digital libraries and scholarly communication infrastructures. All code and data are publicly available on GitHub https://github.com/zhiyintan/SOFT.

Semantically Orthogonal Framework for Citation Classification: Disentangling Intent and Content

TL;DR

Evaluation with both zero-shot and fine-tuned Large Language Models demonstrates that SOFT enables higher agreement between human annotators and LLMs, and supports stronger classification performance and robust cross-domain generalization compared to ACL-ARC and SciCite annotation frameworks.

Abstract

Understanding the role of citations is essential for research assessment and citation-aware digital libraries. However, existing citation classification frameworks often conflate citation intent (why a work is cited) with cited content type (what part is cited), limiting their effectiveness in auto classification due to a dilemma between fine-grained type distinctions and practical classification reliability. We introduce SOFT, a Semantically Orthogonal Framework with Two dimensions that explicitly separates citation intent from cited content type, drawing inspiration from semantic role theory. We systematically re-annotate the ACL-ARC dataset using SOFT and release a cross-disciplinary test set sampled from ACT2. Evaluation with both zero-shot and fine-tuned Large Language Models demonstrates that SOFT enables higher agreement between human annotators and LLMs, and supports stronger classification performance and robust cross-domain generalization compared to ACL-ARC and SciCite annotation frameworks. These results confirm SOFT's value as a clear, reusable annotation standard, improving clarity, consistency, and generalizability for digital libraries and scholarly communication infrastructures. All code and data are publicly available on GitHub https://github.com/zhiyintan/SOFT.
Paper Structure (34 sections, 3 figures, 6 tables)

This paper contains 34 sections, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Citation Intent Classification Frameworks
  4. Citation Intent Classification (CIC) Models
  5. Cross-Domain Generalizability
  6. The Applicability of Re-Annotating Datasets
  7. SOFT: A Semantically Orthogonal Framework with Two Dimensions
  8. Limitations of Existing Citation Classification Frameworks
  9. (1) Dimensional Entanglement: Conflating Cited Content Type and Citation Intent.
  10. (2) Perspective Ambiguity: Authorial Commitment and Discourse Voice.
  11. (3) Lack of a Functionally Grounded Intention Framework.
  12. The SOFT Citation Annotation Framework
  13. Dimension 1: Cited Content Type
  14. Dimension 2: Citation Intent
  15. Design Rationale and Applications.
  16. ...and 19 more sections

Figures (3)

  • Figure 1: Types of statistics for the re-annotated dataset.
  • Figure 2: Comparison of in-domain and cross-domain macro $F_1$ scores across four annotation frameworks. Each subplot corresponds to a model type, with blue circles denoting in-domain and red crosses indicating cross-domain performance.
  • Figure 3: Grid of radar plots (4 frameworks $\times$ 6 models) showing per-class F1-scores. Blue lines/areas represent In-Domain F1 scores, and red lines/areas represent Cross-Domain F1 scores. Each axis corresponds to a class label within the respective framework. F1-scores range from 0 (center) to 1 (outer edge).