Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SEBERTNets: Sequence Enhanced BERT Networks for Event Entity Extraction Tasks Oriented to the Finance Field

Congqing He, Xiangyu Zhu, Yuquan Le, Yuzhong Liu, Jianhong Yin

TL;DR

This study tackles finance-focused event entity extraction, a task requiring both high precision and comprehensive recall when multiple entities appear in text. It introduces SEBERTNets, a sequence-enhanced extension of BERT that adds a sequence-layer component (BiLSTM-based) to capture order-dependent semantics, and its multi-channel recall variant HSEBERTNets, designed to recall all pertinent event entities. The models are validated on the CCKS 2019 finance dataset, using a char-based Chinese BERT backbone and SWATS optimization (Adam followed by SGD), achieving substantial improvements over a BERT baseline (F1 ~0.905 for SEBERTNets and ~0.934 for HSEBERTNets in recall-rich settings). The results, together with qualitative visualizations, demonstrate the effectiveness of integrating sequence-level semantics and multi-channel recall for finance-oriented event entity extraction, with practical implications for investment analysis and asset management.

Abstract

Event extraction lies at the cores of investment analysis and asset management in the financial field, and thus has received much attention. The 2019 China conference on knowledge graph and semantic computing (CCKS) challenge sets up a evaluation competition for event entity extraction task oriented to the finance field. In this task, we mainly focus on how to extract the event entity accurately, and recall all the corresponding event entity effectively. In this paper, we propose a novel model, Sequence Enhanced BERT Networks (SEBERTNets for short), which can inherit the advantages of the BERT,and while capturing sequence semantic information. In addition, motivated by recommendation system, we propose Hybrid Sequence Enhanced BERT Networks (HSEBERTNets for short), which uses a multi-channel recall method to recall all the corresponding event entity. The experimental results show that, the F1 score of SEBERTNets is 0.905 in the first stage, and the F1 score of HSEBERTNets is 0.934 in the first stage, which demonstarate the effectiveness of our methods.

SEBERTNets: Sequence Enhanced BERT Networks for Event Entity Extraction Tasks Oriented to the Finance Field

TL;DR

This study tackles finance-focused event entity extraction, a task requiring both high precision and comprehensive recall when multiple entities appear in text. It introduces SEBERTNets, a sequence-enhanced extension of BERT that adds a sequence-layer component (BiLSTM-based) to capture order-dependent semantics, and its multi-channel recall variant HSEBERTNets, designed to recall all pertinent event entities. The models are validated on the CCKS 2019 finance dataset, using a char-based Chinese BERT backbone and SWATS optimization (Adam followed by SGD), achieving substantial improvements over a BERT baseline (F1 ~0.905 for SEBERTNets and ~0.934 for HSEBERTNets in recall-rich settings). The results, together with qualitative visualizations, demonstrate the effectiveness of integrating sequence-level semantics and multi-channel recall for finance-oriented event entity extraction, with practical implications for investment analysis and asset management.

Abstract

Event extraction lies at the cores of investment analysis and asset management in the financial field, and thus has received much attention. The 2019 China conference on knowledge graph and semantic computing (CCKS) challenge sets up a evaluation competition for event entity extraction task oriented to the finance field. In this task, we mainly focus on how to extract the event entity accurately, and recall all the corresponding event entity effectively. In this paper, we propose a novel model, Sequence Enhanced BERT Networks (SEBERTNets for short), which can inherit the advantages of the BERT,and while capturing sequence semantic information. In addition, motivated by recommendation system, we propose Hybrid Sequence Enhanced BERT Networks (HSEBERTNets for short), which uses a multi-channel recall method to recall all the corresponding event entity. The experimental results show that, the F1 score of SEBERTNets is 0.905 in the first stage, and the F1 score of HSEBERTNets is 0.934 in the first stage, which demonstarate the effectiveness of our methods.
Paper Structure (14 sections, 3 equations, 2 figures, 1 table)

This paper contains 14 sections, 3 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Our Methods
  3. Input Layer
  4. BERT Layer
  5. Sequence Layer
  6. Output Layer
  7. Optimization
  8. Experiments
  9. Dataset
  10. Evaluation
  11. Hyper-parameter Setting
  12. Results
  13. Main Results
  14. Visualization

Figures (2)

  • Figure 1: The architecture of SEBERTNets.
  • Figure 2: Visualization of BERT layer.