Assisted Debate Builder with Large Language Models

TL;DR

This work tackles the challenge of constructing high-quality argumentation frameworks from real-world debates by applying relation-based argument mining (RBAM) with large language models. It introduces ADBL2, an open-source debate builder that imports Kialo debates, verifies existing $attack$/$support$ relations, and assists in creating new arguments, facilitated by a fine-tuned Mistral-7B model for RBAM. A key contribution is a PEFT-based fine-tuning pipeline yielding an average macro F1-score of $90.59\%$ across diverse domains, along with an open-source release of the model and tool. The approach enables cross-domain, resource-efficient RBAM suitable for interactive debate tooling and automatic argument curation, with plans for broader generalization and future enhancements.

Abstract

We introduce ADBL2, an assisted debate builder tool. It is based on the capability of large language models to generalise and perform relation-based argument mining in a wide-variety of domains. It is the first open-source tool that leverages relation-based mining for (1) the verification of pre-established relations in a debate and (2) the assisted creation of new arguments by means of large language models. ADBL2 is highly modular and can work with any open-source large language models that are used as plugins. As a by-product, we also provide the first fine-tuned Mistral-7B large language model for relation-based argument mining, usable by ADBL2, which outperforms existing approaches for this task with an overall F1-score of 90.59% across all domains.

Figures (2)

• Figure 1: Representation of the architecture of the ADBL2 tool.
• Figure 2: Plot of the loss ($y$-axis) on the training (orange) and test (blue) datasets during the fine-tuning per training iteration ($x$-axis).