Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SLO-Conditioned Action Routing for Retrieval-Augmented Generation: Objective Ablation and Failure Modes

Bharath Nunepalli

TL;DR

This work treats retrieval-augmented generation as a per-query control problem where a small discrete action set determines retrieval depth and prompting mode (guarded vs. auto) or abstention, aiming to satisfy service-level objectives. An offline logged dataset from SQuAD 2.0 enables two simple policy learners, Argmax-CE and Argmax-CE-WT, to select actions under two SLO profiles: quality_first and cheap. Experiments show that strong fixed baselines often dominate, with modest gains from learned policies under quality-focused SLOs, while cheap SLOs can induce refusal collapse, highlighting the risk of reward design. The paper emphasizes failure modes, reporting conventions, and a reproducible testbed for SLO-aware RAG control rather than proposing new retrievers or LLMs. The findings illustrate the importance of careful reward shaping and system-level thinking when routing RAG pipelines to meet diverse operational objectives.

Abstract

Retrieval-augmented generation (RAG) introduces a practical control problem: retrieval depth and generation behavior must be chosen per query to satisfy service-level objectives (SLOs) such as cost, refusal rate, and hallucination risk. This work models per-query control as a small discrete action: choose a retrieval depth and a generation mode (guarded vs. auto), or refuse. An offline logged dataset is constructed from SQuAD 2.0 by executing each action and recording accuracy, token cost, hallucination/refusal indicators, and an SLO-weighted reward. Two simple policy-learning objectives are evaluated: supervised classification of the per-state best action (Argmax-CE) and a reward-weighted variant (Argmax-CE-WT). Across the evaluated settings, a strong fixed baseline (low k, guarded prompting) performs competitively; learned policies mainly provide additional cost savings under a quality-focused SLO and can exhibit refusal collapse under a cheap SLO when refusal is heavily rewarded. The contribution is a reproducible case study of SLO-aware control for RAG pipelines, emphasizing failure modes and reporting conventions rather than proposing a new retriever or language model.

SLO-Conditioned Action Routing for Retrieval-Augmented Generation: Objective Ablation and Failure Modes

TL;DR

This work treats retrieval-augmented generation as a per-query control problem where a small discrete action set determines retrieval depth and prompting mode (guarded vs. auto) or abstention, aiming to satisfy service-level objectives. An offline logged dataset from SQuAD 2.0 enables two simple policy learners, Argmax-CE and Argmax-CE-WT, to select actions under two SLO profiles: quality_first and cheap. Experiments show that strong fixed baselines often dominate, with modest gains from learned policies under quality-focused SLOs, while cheap SLOs can induce refusal collapse, highlighting the risk of reward design. The paper emphasizes failure modes, reporting conventions, and a reproducible testbed for SLO-aware RAG control rather than proposing new retrievers or LLMs. The findings illustrate the importance of careful reward shaping and system-level thinking when routing RAG pipelines to meet diverse operational objectives.

Abstract

Retrieval-augmented generation (RAG) introduces a practical control problem: retrieval depth and generation behavior must be chosen per query to satisfy service-level objectives (SLOs) such as cost, refusal rate, and hallucination risk. This work models per-query control as a small discrete action: choose a retrieval depth and a generation mode (guarded vs. auto), or refuse. An offline logged dataset is constructed from SQuAD 2.0 by executing each action and recording accuracy, token cost, hallucination/refusal indicators, and an SLO-weighted reward. Two simple policy-learning objectives are evaluated: supervised classification of the per-state best action (Argmax-CE) and a reward-weighted variant (Argmax-CE-WT). Across the evaluated settings, a strong fixed baseline (low k, guarded prompting) performs competitively; learned policies mainly provide additional cost savings under a quality-focused SLO and can exhibit refusal collapse under a cheap SLO when refusal is heavily rewarded. The contribution is a reproducible case study of SLO-aware control for RAG pipelines, emphasizing failure modes and reporting conventions rather than proposing a new retriever or language model.
Paper Structure (30 sections, 1 equation, 3 figures, 1 table)

This paper contains 30 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem setup
  4. RAG action space
  5. SLO profiles and reward
  6. State representation
  7. Methods
  8. Offline log generation
  9. Policy training objectives
  10. Retriever, evaluation, and prompt templates
  11. Retrieval corpus and hit rate.
  12. Prompting modes.
  13. Experiments
  14. Dataset and evaluation
  15. Reproducibility and artifacts
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Action distribution of learned policies under each SLO and objective. Cost-focused settings can collapse to refusal-heavy policies.
  • Figure 2: Average token cost vs. accuracy for learned policies and best fixed-action baselines.
  • Figure 3: Average reward for best fixed actions vs. learned policies under each condition.