Slot Filling as a Reasoning Task for SpeechLLMs

TL;DR

The paper investigates incorporating chain-of-thought reasoning into speech-based LLMs for end-to-end slot filling. By constructing instruction-based and reasoning-style datasets and applying supervised fine-tuning with PEFT, the authors evaluate regular, reasoning, and hybrid speechLLMs across a spectrum of foundation LLMs. They find that reasoning traces can improve performance in several configurations, but reasoning-focused models may degrade general language understanding; hybrid training that merges direct and reasoning outputs often yields the strongest results, especially for medium-scale bases. The work provides practical guidance for designing efficient, flexible speech understanding systems that balance instruction-following and reasoning capabilities to achieve robust slot-filling performance.

Abstract

We propose integration of reasoning into speech large language models (speechLLMs) for the end-to-end slot-filling task. Inspired by the recent development of reasoning LLMs, we use a chain-of-thought framework to decompose the slot-filling task into multiple reasoning steps, create a reasoning dataset and apply the supervised fine-tuning strategy to a speechLLM. We distinguish between regular and reasoning speechLLMs and experiment with different types and sizes of LLMs as their text foundation models. We demonstrate performance improvements by introducing reasoning (intermediate) steps. However, we show that a reasoning textual LLM developed mainly for math, logic and coding domains might be inferior as a foundation model for a reasoning speechLLM. We further show that hybrid speechLLMs, built on a hybrid text foundation LLM and fine-tuned to preserve both direct and reasoning modes of operation, have better performance than those fine-tuned employing only one mode of operation.

Figures (3)

• Figure 1: SpeechLLM architecture
• Figure 2: SpeechLLM types
• Figure 3: Comparison of regular and reasoning-style slot-filling responses.