Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FLAG: Financial Long Document Classification via AMR-based GNN

Bolun "Namir" Xia, Aparna Gupta, Mohammed J. Zaki

TL;DR

This work proposes FLAG: Financial Long document classification via AMR-based GNN, an AMR graph based framework to generate document-level embeddings for long financial document classification and finds that the AMR-based approach outperforms fine-tuning LLMs directly on text in predicting stock price movement trends at different time horizons in both datasets.

Abstract

The advent of large language models (LLMs) has initiated much research into their various financial applications. However, in applying LLMs on long documents, semantic relations are not explicitly incorporated, and a full or arbitrarily sparse attention operation is employed. In recent years, progress has been made in Abstract Meaning Representation (AMR), which is a graph-based representation of text to preserve its semantic relations. Since AMR can represent semantic relationships at a deeper level, it can be beneficially utilized by graph neural networks (GNNs) for constructing effective document-level graph representations built upon LLM embeddings to predict target metrics in the financial domain. We propose FLAG: Financial Long document classification via AMR-based GNN, an AMR graph based framework to generate document-level embeddings for long financial document classification. We construct document-level graphs from sentence-level AMR graphs, endow them with specialized LLM word embeddings in the financial domain, apply a deep learning mechanism that utilizes a GNN, and examine the efficacy of our AMR-based approach in predicting labeled target data from long financial documents. Extensive experiments are conducted on a dataset of quarterly earnings calls transcripts of companies in various sectors of the economy, as well as on a corpus of more recent earnings calls of companies in the S&P 1500 Composite Index. We find that our AMR-based approach outperforms fine-tuning LLMs directly on text in predicting stock price movement trends at different time horizons in both datasets. Our work also outperforms previous work utilizing document graphs and GNNs for text classification.

FLAG: Financial Long Document Classification via AMR-based GNN

TL;DR

This work proposes FLAG: Financial Long document classification via AMR-based GNN, an AMR graph based framework to generate document-level embeddings for long financial document classification and finds that the AMR-based approach outperforms fine-tuning LLMs directly on text in predicting stock price movement trends at different time horizons in both datasets.

Abstract

The advent of large language models (LLMs) has initiated much research into their various financial applications. However, in applying LLMs on long documents, semantic relations are not explicitly incorporated, and a full or arbitrarily sparse attention operation is employed. In recent years, progress has been made in Abstract Meaning Representation (AMR), which is a graph-based representation of text to preserve its semantic relations. Since AMR can represent semantic relationships at a deeper level, it can be beneficially utilized by graph neural networks (GNNs) for constructing effective document-level graph representations built upon LLM embeddings to predict target metrics in the financial domain. We propose FLAG: Financial Long document classification via AMR-based GNN, an AMR graph based framework to generate document-level embeddings for long financial document classification. We construct document-level graphs from sentence-level AMR graphs, endow them with specialized LLM word embeddings in the financial domain, apply a deep learning mechanism that utilizes a GNN, and examine the efficacy of our AMR-based approach in predicting labeled target data from long financial documents. Extensive experiments are conducted on a dataset of quarterly earnings calls transcripts of companies in various sectors of the economy, as well as on a corpus of more recent earnings calls of companies in the S&P 1500 Composite Index. We find that our AMR-based approach outperforms fine-tuning LLMs directly on text in predicting stock price movement trends at different time horizons in both datasets. Our work also outperforms previous work utilizing document graphs and GNNs for text classification.
Paper Structure (16 sections, 4 equations, 3 figures, 11 tables)

This paper contains 16 sections, 4 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. The FLAG Approach
  4. Sentence AMR Parsing
  5. Document-level Graph Construction
  6. GNN Model Training and Fine-Tuning
  7. Empirical Evaluation
  8. Dataset and Metrics
  9. Methods
  10. Comparative Results
  11. Ablation Studies
  12. Efficacy of AMR document-level graphs: a comparison between StockGNN and Gated GNN on FLAG graphs
  13. Efficacy of attention-based GATv2: a comparison of different GNNs applied on FLAG graphs
  14. Various configurations of GATv2: number of layers, attention heads, and different hidden dimensions
  15. Case Study
  16. ...and 1 more sections

Figures (3)

  • Figure 1: An example of the AMR graph for the sentence: an investment in our common stock is subject to risks inherent to our business.
  • Figure 2: FLAG Architecture: Each document is parsed into sentences, which are converted into sentence AMR graphs. Using our hierarchical approach, we combine them into the document-level graph, which is endowed with the word embeddings and we then apply the GNN model to generate the final document virtual node embedding. This embedding is then passed through two fully connected linear layers to predict the target output. F: forward propagation; B: backward propagation.
  • Figure 3: Document-level graph construction.