Enhancing Language Models for Financial Relation Extraction with Named Entities and Part-of-Speech
Menglin Li, Kwan Hui Lim
TL;DR
The paper tackles financial relation extraction (FinRE) by augmenting a RoBERTa-based model with linguistic signals from Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. The authors propose replacing textual content with NER tokens and concatenating POS tokens (TrNP) to enrich input representations, and conduct extensive ablations on the REFinD dataset. Their results show state-of-the-art Micro-F1 and Macro-F1 scores, with Micro-F1 of 0.7721 and Macro-F1 of 0.5507, and demonstrate that NER replacement yields larger gains than POS. This approach offers a simple yet effective way to boost FinRE performance and suggests broad applicability to domain-specific relation extraction tasks.
Abstract
The Financial Relation Extraction (FinRE) task involves identifying the entities and their relation, given a piece of financial statement/text. To solve this FinRE problem, we propose a simple but effective strategy that improves the performance of pre-trained language models by augmenting them with Named Entity Recognition (NER) and Part-Of-Speech (POS), as well as different approaches to combine these information. Experiments on a financial relations dataset show promising results and highlights the benefits of incorporating NER and POS in existing models. Our dataset and codes are available at https://github.com/kwanhui/FinRelExtract.