Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reliability-Aware Determinantal Point Processes for Robust Informative Data Selection in Large Language Models

Ahmad Sarlak, Abolfazl Razi

TL;DR

This work tackles the challenge of selecting informative data subsets for large language model workflows when candidate units may randomly dropout due to storage, communication, or processing unreliability. The authors identify ill-posedness of naive dropout-augmented log-determinant objectives and rectify it with a minimal regularization, yielding a reliability-aware diversity objective that cleanly decomposes into a geometric diversity term and an additive per-item reliability term. They model the problem as a combinatorial semi-bandit (RA-kDPP) and design a KL-UCB–style online algorithm with provable regret bounds and matching information-theoretic lower bounds, while enabling online learning of unknown reliabilities from semi-bandit feedback. Empirically, ProbDPP improves robustness and accuracy in multi-source prompting and retrieval-augmented QA tasks under stochastic availability, outperforming both reliability-only and diversity-only baselines. The approach provides a principled foundation for resilient data selection in LLM fine-tuning, prompt compression, and RAG systems under unreliable data access, with practical impact on efficiency and performance in real-world deployments.

Abstract

Informative data selection is a key requirement for large language models (LLMs) to minimize the amount of data required for fine-tuning, network distillation, and token pruning, enabling fast and efficient deployment, especially under computational and communication constraints. Traditional subset selection methods, including those based on Determinantal Point Processes (DPP), focus on maximizing diversity but assume that selected data batches are always available error-free. This presumption prohibits their use under partial storage outage, imperfect communication, and stochastic access failures. Furthermore, we show that the original formulation collapses under such conditions. To address this gap, we introduce ProbDPP, a novel reliability-aware implementation of k-DPP that accounts for probabilistic data access by recasting the objective function with a regularization term that remains well-posed and decomposes into a geometric diversity term and unreliability cost. The resulting objective facilitates robust selection of diverse data batches under uncertainty. Furthermore, we frame this reliability-aware diversity maximization as a combinatorial semi-bandit problem and propose a UCB-style algorithm to efficiently learn the unknown reliability online. Theoretical analysis provides regret bounds for the proposed approach, ensuring performance guarantees.

Reliability-Aware Determinantal Point Processes for Robust Informative Data Selection in Large Language Models

TL;DR

This work tackles the challenge of selecting informative data subsets for large language model workflows when candidate units may randomly dropout due to storage, communication, or processing unreliability. The authors identify ill-posedness of naive dropout-augmented log-determinant objectives and rectify it with a minimal regularization, yielding a reliability-aware diversity objective that cleanly decomposes into a geometric diversity term and an additive per-item reliability term. They model the problem as a combinatorial semi-bandit (RA-kDPP) and design a KL-UCB–style online algorithm with provable regret bounds and matching information-theoretic lower bounds, while enabling online learning of unknown reliabilities from semi-bandit feedback. Empirically, ProbDPP improves robustness and accuracy in multi-source prompting and retrieval-augmented QA tasks under stochastic availability, outperforming both reliability-only and diversity-only baselines. The approach provides a principled foundation for resilient data selection in LLM fine-tuning, prompt compression, and RAG systems under unreliable data access, with practical impact on efficiency and performance in real-world deployments.

Abstract

Informative data selection is a key requirement for large language models (LLMs) to minimize the amount of data required for fine-tuning, network distillation, and token pruning, enabling fast and efficient deployment, especially under computational and communication constraints. Traditional subset selection methods, including those based on Determinantal Point Processes (DPP), focus on maximizing diversity but assume that selected data batches are always available error-free. This presumption prohibits their use under partial storage outage, imperfect communication, and stochastic access failures. Furthermore, we show that the original formulation collapses under such conditions. To address this gap, we introduce ProbDPP, a novel reliability-aware implementation of k-DPP that accounts for probabilistic data access by recasting the objective function with a regularization term that remains well-posed and decomposes into a geometric diversity term and unreliability cost. The resulting objective facilitates robust selection of diverse data batches under uncertainty. Furthermore, we frame this reliability-aware diversity maximization as a combinatorial semi-bandit problem and propose a UCB-style algorithm to efficiently learn the unknown reliability online. Theoretical analysis provides regret bounds for the proposed approach, ensuring performance guarantees.
Paper Structure (28 sections, 3 theorems, 61 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 28 sections, 3 theorems, 61 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Problem Formulation
  5. Why the naïve dropout-aware log-det collapses.
  6. Reliability-aware ProbDPP Objective
  7. Interpretation.
  8. Online objective.
  9. RA-kDPP: A Combinatorial Semi-Bandit Formulation
  10. Semi-bandit feedback and estimation
  11. Action selection with KL-UCB optimism
  12. Performance Analysis
  13. Benchmark and Regret
  14. Lower Bound for ProbDPP
  15. Restricted instance.
  16. ...and 13 more sections

Key Result

Lemma 3.1

Fix any subset $L\subseteq[N]$ such that there exists $i\in L$ with $\alpha_i<1$. Let $z_i\sim\mathrm{Bernoulli}(\alpha_i)$ be independent and define $\Sigma_{L,L}(z)=M_L(z)\,G_{L,L}\,M_L(z)$ with $M_L(z)=\mathrm{diag}(z_i:i\in L)$. Then

Figures (2)

  • Figure 1: Robustness to stochastic source availability. Performance vs. survival average probability $\alpha$ with $N=10$ and $K=5$. Error bars show mean $\pm$ std.
  • Figure 2: Effect of selection budget under stochastic availability. Performance vs. selection budget $K$ with $N=10$ candidate sources. Error bars show mean $\pm$ std.

Theorems & Definitions (3)

  • Lemma 3.1: Ill-posedness under dropouts
  • Lemma 3.2: Exact decomposition of the regularized objective
  • Theorem 4.1: Regret of ProbDPP