Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Logits-Constrained Framework with RoBERTa for Ancient Chinese NER

Wenjie Hua, Shenghan Xu

TL;DR

The paper tackles Ancient Chinese NER by addressing label-sequence errors arising from complex semantics. It introduces a two-stage Logits-Constrained framework that uses GujiRoBERTa for contextual encoding and differentiable decoding with a BMES-transition constraint matrix to enforce valid tag sequences, eliminating the need for separate CRF post-processing. Empirical results on EvaHan 2025 show LC outperforms traditional CRF-based methods, with greater gains in high-label or large-data settings, while BiLSTM integration tends to hurt performance. A data-driven model-selection criterion further guides when to use LC alone versus CRF+LC, enhancing practical applicability for real-world ancient Chinese NLP tasks, albeit with some limitations related to constraint design and inference overhead.

Abstract

This paper presents a Logits-Constrained (LC) framework for Ancient Chinese Named Entity Recognition (NER), evaluated on the EvaHan 2025 benchmark. Our two-stage model integrates GujiRoBERTa for contextual encoding and a differentiable decoding mechanism to enforce valid BMES label transitions. Experiments demonstrate that LC improves performance over traditional CRF and BiLSTM-based approaches, especially in high-label or large-data settings. We also propose a model selection criterion balancing label complexity and dataset size, providing practical guidance for real-world Ancient Chinese NLP tasks.

Logits-Constrained Framework with RoBERTa for Ancient Chinese NER

TL;DR

The paper tackles Ancient Chinese NER by addressing label-sequence errors arising from complex semantics. It introduces a two-stage Logits-Constrained framework that uses GujiRoBERTa for contextual encoding and differentiable decoding with a BMES-transition constraint matrix to enforce valid tag sequences, eliminating the need for separate CRF post-processing. Empirical results on EvaHan 2025 show LC outperforms traditional CRF-based methods, with greater gains in high-label or large-data settings, while BiLSTM integration tends to hurt performance. A data-driven model-selection criterion further guides when to use LC alone versus CRF+LC, enhancing practical applicability for real-world ancient Chinese NLP tasks, albeit with some limitations related to constraint design and inference overhead.

Abstract

This paper presents a Logits-Constrained (LC) framework for Ancient Chinese Named Entity Recognition (NER), evaluated on the EvaHan 2025 benchmark. Our two-stage model integrates GujiRoBERTa for contextual encoding and a differentiable decoding mechanism to enforce valid BMES label transitions. Experiments demonstrate that LC improves performance over traditional CRF and BiLSTM-based approaches, especially in high-label or large-data settings. We also propose a model selection criterion balancing label complexity and dataset size, providing practical guidance for real-world Ancient Chinese NLP tasks.
Paper Structure (18 sections, 5 equations, 1 figure, 6 tables)

This paper contains 18 sections, 5 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. RoBERTa
  4. Transition Constraints in Sequence Labeling
  5. Method
  6. Pre-processing
  7. Framework
  8. Stage 1: Contextual Encoding with GujiRoBERTa
  9. Stage 2: Logits-Constrained Decoding
  10. Experiments
  11. Experimental Environment
  12. Parameter Regulation
  13. GujiRoBERTa
  14. Cross-Comparison
  15. Dataset Expansion
  16. ...and 3 more sections

Figures (1)

  • Figure 1: Framework Overview