Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MEANT: Multimodal Encoder for Antecedent Information

Benjamin Iyoya Irving, Annika Marie Schoene

TL;DR

The MEANT model, a Multimodal Encoder for Antecedent information and a new dataset called TempStock, which consists of price, Tweets, and graphical data with over a million Tweets from all of the companies in the S&P 500 Index are introduced.

Abstract

The stock market provides a rich well of information that can be split across modalities, making it an ideal candidate for multimodal evaluation. Multimodal data plays an increasingly important role in the development of machine learning and has shown to positively impact performance. But information can do more than exist across modes -- it can exist across time. How should we attend to temporal data that consists of multiple information types? This work introduces (i) the MEANT model, a Multimodal Encoder for Antecedent information and (ii) a new dataset called TempStock, which consists of price, Tweets, and graphical data with over a million Tweets from all of the companies in the S&P 500 Index. We find that MEANT improves performance on existing baselines by over 15%, and that the textual information affects performance far more than the visual information on our time-dependent task from our ablation study.

MEANT: Multimodal Encoder for Antecedent Information

TL;DR

The MEANT model, a Multimodal Encoder for Antecedent information and a new dataset called TempStock, which consists of price, Tweets, and graphical data with over a million Tweets from all of the companies in the S&P 500 Index are introduced.

Abstract

The stock market provides a rich well of information that can be split across modalities, making it an ideal candidate for multimodal evaluation. Multimodal data plays an increasingly important role in the development of machine learning and has shown to positively impact performance. But information can do more than exist across modes -- it can exist across time. How should we attend to temporal data that consists of multiple information types? This work introduces (i) the MEANT model, a Multimodal Encoder for Antecedent information and (ii) a new dataset called TempStock, which consists of price, Tweets, and graphical data with over a million Tweets from all of the companies in the S&P 500 Index. We find that MEANT improves performance on existing baselines by over 15%, and that the textual information affects performance far more than the visual information on our time-dependent task from our ablation study.

Paper Structure

This paper contains 35 sections, 22 equations, 4 figures, 22 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Models for Financial Twitter Data
  4. Financial Twitter Datasets
  5. TempStock Dataset
  6. TempStock
  7. MEANT
  8. Encoder Only
  9. Token and Patch Embeddings
  10. Positional Encoding
  11. Temporal Encoder
  12. Experiments
  13. Fine-tuning on downstream tasks
  14. TempStock
  15. Stocknet
  16. ...and 20 more sections

Figures (4)

  • Figure 1: An example of a graph from our MACD data, which displays the MACD (in blue) and the signal line (in red) for MMM (3M) over a 26 day period. Along the x-axis, we see 11 of the dates listed, and the the y-axis shows the value of the aforementioned indicators. In each bar lies the value of the MACD histogram, which is the difference between the MACD (blue) and the Signal line (red).
  • Figure 2: A schematic overview of the MEANT architecture. As seen in the diagram, the output of the language encoder is processed in two different variants: sequence projection, and mean pooling.
  • Figure 3: Confusion matrix for MEANT-XL on TempStock
  • Figure 4: Confusion matrix for TEANet on TempStock