Argumentative Reasoning with Language Models on Non-factorized Case Bases

TL;DR

The paper tackles case-based reasoning with non-factorized case bases by proposing AAM-CBR, a framework that uses language-model agents to determine case coverage and extract factors from new cases within an abstract-argumentation (AA-CBR) setting. It demonstrates how this hybrid, symbolic-LLM approach preserves privacy and improves interpretability while maintaining predictive performance, especially as the number of factors grows. Through synthetic credit-card decision experiments, the authors show AAM-CBR can outperform single-prompt baselines when new cases are rich in factors, highlighting the value of combining symbolic reasoning with LLMs in high-factor regimes. The work also identifies challenges in coverage accuracy and factor extraction stability, motivating further research in incremental factor learning and robust prompt design.

Abstract

In this paper, we investigate how language models can perform case-based reasoning (CBR) on non-factorized case bases. We introduce a novel framework, argumentative agentic models for case-based reasoning (AAM-CBR), which extends abstract argumentation for case-based reasoning (AA-CBR). Unlike traditional approaches that require factorization of previous cases, AAM-CBR leverages language models to determine case coverage and extract factors based on new cases. This enables factor-based reasoning without exposing or preprocessing previous cases, thus improving both flexibility and privacy. We also present initial experiments to assess AAM-CBR performance by comparing the proposed framework with a baseline that uses a single-prompt approach to incorporate both new and previous cases. The experiments are conducted based on a synthetic credit card application dataset. The result shows that AAM-CBR surpasses the baseline only when the new case contains a richer set of factors. The finding indicates that language models can handle case-based reasoning with a limited number of factors, but face challenges as the number of factors increase. Consequently, integrating symbolic reasoning with language models, as implemented in AAM-CBR, is crucial for effectively handling cases involving many factors.

Figures (4)

• Figure 1: Corresponding AA framework with default outcome $'{0}\mathchar''$
• Figure 2: Corresponding AA framework with default outcome $'{1}\mathchar''$
• Figure 3: Overview of AAM-CBR
• Figure 4: Comparative performance on case relevance determination and case factor extraction tasks