Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Case-Based Reasoning Approach for Solving Financial Question Answering

Yikyung Kim, Jay-Yoon Lee

TL;DR

This work addresses the challenge of numerical reasoning in financial QA by introducing a Case-Based Reasoning (CBR) framework that retrieves similar question–program cases to guide program generation. The approach builds on FinQANet's context-retrieval and program-generation paradigm, augmenting it with two retrieval architectures (Bi-Encoder and Cross-Encoder) and two integration schemes (Concatenation and Separate Encoder). Experimental results show that using gold cases yields substantial gains in program accuracy and that a high-quality retriever is critical to the overall performance, with dual-stage retrieval offering a path to improved precision. The study highlights the potential of leveraging similar cases to improve logical step generation in multi-modal financial documents and points to retriever quality as the key area for future improvement.

Abstract

Measuring a machine's understanding of human language often involves assessing its reasoning skills, i.e. logical process of deriving answers to questions. While recent language models have shown remarkable proficiency in text based tasks, their efficacy in complex reasoning problems involving heterogeneous information such as text, tables, and numbers remain uncertain. Addressing this gap, FinQA introduced a numerical reasoning dataset for financial documents and simultaneously proposed a program generation approach . Our investigation reveals that half of the errors (48%) stem from incorrect operations being generated. To address this issue, we propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR), an artificial intelligence paradigm that provides problem solving guidance by offering similar cases (i.e. similar questions and corresponding logical programs). Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information. Through experiments on the FinQA dataset, we demonstrate competitive performance of our approach and additionally show that by expanding case repository, we can help solving complex multi step programs which FinQA showed weakness of.

Case-Based Reasoning Approach for Solving Financial Question Answering

TL;DR

This work addresses the challenge of numerical reasoning in financial QA by introducing a Case-Based Reasoning (CBR) framework that retrieves similar question–program cases to guide program generation. The approach builds on FinQANet's context-retrieval and program-generation paradigm, augmenting it with two retrieval architectures (Bi-Encoder and Cross-Encoder) and two integration schemes (Concatenation and Separate Encoder). Experimental results show that using gold cases yields substantial gains in program accuracy and that a high-quality retriever is critical to the overall performance, with dual-stage retrieval offering a path to improved precision. The study highlights the potential of leveraging similar cases to improve logical step generation in multi-modal financial documents and points to retriever quality as the key area for future improvement.

Abstract

Measuring a machine's understanding of human language often involves assessing its reasoning skills, i.e. logical process of deriving answers to questions. While recent language models have shown remarkable proficiency in text based tasks, their efficacy in complex reasoning problems involving heterogeneous information such as text, tables, and numbers remain uncertain. Addressing this gap, FinQA introduced a numerical reasoning dataset for financial documents and simultaneously proposed a program generation approach . Our investigation reveals that half of the errors (48%) stem from incorrect operations being generated. To address this issue, we propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR), an artificial intelligence paradigm that provides problem solving guidance by offering similar cases (i.e. similar questions and corresponding logical programs). Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information. Through experiments on the FinQA dataset, we demonstrate competitive performance of our approach and additionally show that by expanding case repository, we can help solving complex multi step programs which FinQA showed weakness of.
Paper Structure (34 sections, 3 equations, 4 figures, 2 tables)

This paper contains 34 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Overview of FinQANet
  4. Context Retriever
  5. Program Generator
  6. Program
  7. Exploring Case-Based Reasoning (CBR)
  8. Defining Gold Cases
  9. Related Works
  10. Text-Based Question Answering
  11. Table-Based Question Answering
  12. Question Answering Requiring Numerical Reasoning
  13. Methodology
  14. Retrieving Cases
  15. Bi-Encoder Architecture
  16. ...and 19 more sections

Figures (4)

  • Figure 1: FinQANet Program Generator
  • Figure 2: Overview of the Enhanced CBR Architecture. This diagram illustrates the integration of a case retriever into the FinQA baseline model. The case retriever is designed to identify relevant cases from the dataset that share similarities in question and program. These cases, along with the related contexts retrieved from the document, are then fed into the program generator to facilitate the generation of program sequence.
  • Figure 3: Bi-Encoder Architecture: the question and candidate case are encoded separately to generate distinct embeddings.
  • Figure 4: Cross-Encoder Architecture: the question and candidate case are concatenated before encoding to create a single embedding.