Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts

Rajdeep Mukherjee, Abhinav Bohra, Akash Banerjee, Soumya Sharma, Manjunath Hegde, Afreen Shaikh, Shivani Shrivastava, Koustuv Dasgupta, Niloy Ganguly, Saptarshi Ghosh, Pawan Goyal

TL;DR

ECTSum introduces a long-form finance dataset pairing earnings call transcripts with expert-written Reuters bullet summaries, addressing the paucity of long financial-domain benchmarks. The authors propose ECT-BPS, an extractive-then-paraphrase pipeline that first selects salient ECT sentences and then paraphrases them into concise telegram-style bullets, preserving numerical facts. Across automatic and human evaluations, ECT-BPS outperforms strong baselines, highlighting the value of a targeted two-stage approach for highly condensed, numerically dense financial content. The dataset and code are publicly released to foster further research in finance-domain long-document summarization.

Abstract

Despite tremendous progress in automatic summarization, state-of-the-art methods are predominantly trained to excel in summarizing short newswire articles, or documents with strong layout biases such as scientific articles or government reports. Efficient techniques to summarize financial documents, including facts and figures, have largely been unexplored, majorly due to the unavailability of suitable datasets. In this work, we present ECTSum, a new dataset with transcripts of earnings calls (ECTs), hosted by publicly traded companies, as documents, and short experts-written telegram-style bullet point summaries derived from corresponding Reuters articles. ECTs are long unstructured documents without any prescribed length limit or format. We benchmark our dataset with state-of-the-art summarizers across various metrics evaluating the content quality and factual consistency of the generated summaries. Finally, we present a simple-yet-effective approach, ECT-BPS, to generate a set of bullet points that precisely capture the important facts discussed in the calls.

ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts

TL;DR

ECTSum introduces a long-form finance dataset pairing earnings call transcripts with expert-written Reuters bullet summaries, addressing the paucity of long financial-domain benchmarks. The authors propose ECT-BPS, an extractive-then-paraphrase pipeline that first selects salient ECT sentences and then paraphrases them into concise telegram-style bullets, preserving numerical facts. Across automatic and human evaluations, ECT-BPS outperforms strong baselines, highlighting the value of a targeted two-stage approach for highly condensed, numerically dense financial content. The dataset and code are publicly released to foster further research in finance-domain long-document summarization.

Abstract

Despite tremendous progress in automatic summarization, state-of-the-art methods are predominantly trained to excel in summarizing short newswire articles, or documents with strong layout biases such as scientific articles or government reports. Efficient techniques to summarize financial documents, including facts and figures, have largely been unexplored, majorly due to the unavailability of suitable datasets. In this work, we present ECTSum, a new dataset with transcripts of earnings calls (ECTs), hosted by publicly traded companies, as documents, and short experts-written telegram-style bullet point summaries derived from corresponding Reuters articles. ECTs are long unstructured documents without any prescribed length limit or format. We benchmark our dataset with state-of-the-art summarizers across various metrics evaluating the content quality and factual consistency of the generated summaries. Finally, we present a simple-yet-effective approach, ECT-BPS, to generate a set of bullet points that precisely capture the important facts discussed in the calls.
Paper Structure (26 sections, 2 equations, 3 figures, 5 tables)

This paper contains 26 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Dataset
  4. Data Collection
  5. Data Cleaning and Pairing
  6. Statistics and Analysis
  7. The ECT-BPS Framework
  8. The Extractive Module
  9. The Paraphrasing Module
  10. Training and Inference
  11. Experiments and Results
  12. Baselines
  13. Evaluation Metrics
  14. Experimental Setup
  15. Main Results
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Salient unigram distribution in four equally sized segments of the source text. Higher percentages indicate higher unigram overlap. Percentages more than 25 indicate there are repetitions.
  • Figure 2: ECT-BPS: Our Proposed Summarization Framework. It consists of an Extractive Module that is trained to select highly salient sentences from the source document. The Paraphrasing Module is then trained to paraphrase the ECT sentences to the (Reuters) format of target summary sentences.
  • Figure 3: Histogram distribution for human evaluation scores assigned to model-generated summaries.