SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation

TL;DR

SusGen-30K fills a critical gap by providing a balanced, multi-task dataset and a tuned LLM family (SusGen-GPT) for finance and ESG tasks, including ESG-report generation. The authors also introduce TCFD-Bench to evaluate ESG reporting quality and demonstrate that SusGen-GPT achieves near GPT-4 performance with only 7–8B parameters, aided by Retrieval-Augmented Generation (RAG) for extracting ESG content from annual reports. A data-centric fine-tuning pipeline with LoRA/QLoRA, multilingual augmentation, anonymization, and instruction-based formatting enables strong cross-task performance while maintaining efficiency. The work highlights practical, scalable approaches for advancing financial NLP and ESG reporting, and suggests future work to broaden ESG subtasks, diversify report templates, and enhance evaluation. Overall, SusGen-GPT offers a deployable, efficient tool for generating TCFD-compliant ESG reports and supporting ESG-related inquiries in finance.

Abstract

The rapid growth of the financial sector and the rising focus on Environmental, Social, and Governance (ESG) considerations highlight the need for advanced NLP tools. However, open-source LLMs proficient in both finance and ESG domains remain scarce. To address this gap, we introduce SusGen-30K, a category-balanced dataset comprising seven financial NLP tasks and ESG report generation, and propose TCFD-Bench, a benchmark for evaluating sustainability report generation. Leveraging this dataset, we developed SusGen-GPT, a suite of models achieving state-of-the-art performance across six adapted and two off-the-shelf tasks, trailing GPT-4 by only 2% despite using 7-8B parameters compared to GPT-4's 1,700B. Based on this, we propose the SusGen system, integrated with Retrieval-Augmented Generation (RAG), to assist in sustainability report generation. This work demonstrates the efficiency of our approach, advancing research in finance and ESG.

Figures (8)

• Figure 1: An overview of model comparison with both open-source and closed-base baseline models on six financial NLP tasks. The two sub-figures show that our model SusGen-GPT achieves state-of-the-art performance in most benchmarks.
• Figure 2: Overview of the SusGen System Pipeline.
• Figure 3: The data construction pipeline of TCFD-Bench, illustrated with an example extracted and processed from the Wolfspeed_2022.pdf reports.
• Figure 4: The pipeline of SusGen-30K data construction. The process involves collecting data open-source datasets from hugging-face and company reports from TCFD-Hub Database, followed by quality control and various automatic LLMs pre-processing steps to create the final instruction-following format dataset.
• Figure 5: SusGen-30K Category Distribution. Highlight the proportion of data dedicated to each specific task area in financial NLP.