Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning

Junyu Cao, Ruijiang Gao, Esmaeil Keyvanshokooh, Jianhao Ma

TL;DR

The paper tackles trustworthy sequential decision-making in high-stakes domains by coupling algorithmic recourse with contextual bandits and large language models. It introduces GLRB, a Generalized Linear Recourse Bandit, and LIBRA, a Language Model–Informed Bandit Recourse Algorithm, delivering strong theoretical guarantees: a warm-start effect when LLM guidance is reliable, an LLM-effort bound of $O(\log^2 T)$ queries, and robustness to unreliable LLMs. The authors prove matching lower bounds and demonstrate near-optimal performance, with empirical validation on synthetic data and a hypertension management case study using the ACCORD dataset. Collectively, the work shows that recourse-aware, LLM-assisted bandits can achieve improved regret, treatment quality, and sample efficiency while maintaining safety and autonomy in personalized high-stakes decision-making.

Abstract

We introduce a unified framework that seamlessly integrates algorithmic recourse, contextual bandits, and large language models (LLMs) to support sequential decision-making in high-stakes settings such as personalized medicine. We first introduce the recourse bandit problem, where a decision-maker must select both a treatment action and a feasible, minimal modification to mutable patient features. To address this problem, we develop the Generalized Linear Recourse Bandit (GLRB) algorithm. Building on this foundation, we propose LIBRA, a Language Model-Informed Bandit Recourse Algorithm that strategically combines domain knowledge from LLMs with the statistical rigor of bandit learning. LIBRA offers three key guarantees: (i) a warm-start guarantee, showing that LIBRA significantly reduces initial regret when LLM recommendations are near-optimal; (ii) an LLM-effort guarantee, proving that the algorithm consults the LLM only $O(\log^2 T)$ times, where $T$ is the time horizon, ensuring long-term autonomy; and (iii) a robustness guarantee, showing that LIBRA never performs worse than a pure bandit algorithm even when the LLM is unreliable. We further establish matching lower bounds that characterize the fundamental difficulty of the recourse bandit problem and demonstrate the near-optimality of our algorithms. Experiments on synthetic environments and a real hypertension-management case study confirm that GLRB and LIBRA improve regret, treatment quality, and sample efficiency compared with standard contextual bandits and LLM-only benchmarks. Our results highlight the promise of recourse-aware, LLM-assisted bandit algorithms for trustworthy LLM-bandits collaboration in personalized high-stakes decision-making.

LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning

TL;DR

The paper tackles trustworthy sequential decision-making in high-stakes domains by coupling algorithmic recourse with contextual bandits and large language models. It introduces GLRB, a Generalized Linear Recourse Bandit, and LIBRA, a Language Model–Informed Bandit Recourse Algorithm, delivering strong theoretical guarantees: a warm-start effect when LLM guidance is reliable, an LLM-effort bound of queries, and robustness to unreliable LLMs. The authors prove matching lower bounds and demonstrate near-optimal performance, with empirical validation on synthetic data and a hypertension management case study using the ACCORD dataset. Collectively, the work shows that recourse-aware, LLM-assisted bandits can achieve improved regret, treatment quality, and sample efficiency while maintaining safety and autonomy in personalized high-stakes decision-making.

Abstract

We introduce a unified framework that seamlessly integrates algorithmic recourse, contextual bandits, and large language models (LLMs) to support sequential decision-making in high-stakes settings such as personalized medicine. We first introduce the recourse bandit problem, where a decision-maker must select both a treatment action and a feasible, minimal modification to mutable patient features. To address this problem, we develop the Generalized Linear Recourse Bandit (GLRB) algorithm. Building on this foundation, we propose LIBRA, a Language Model-Informed Bandit Recourse Algorithm that strategically combines domain knowledge from LLMs with the statistical rigor of bandit learning. LIBRA offers three key guarantees: (i) a warm-start guarantee, showing that LIBRA significantly reduces initial regret when LLM recommendations are near-optimal; (ii) an LLM-effort guarantee, proving that the algorithm consults the LLM only times, where is the time horizon, ensuring long-term autonomy; and (iii) a robustness guarantee, showing that LIBRA never performs worse than a pure bandit algorithm even when the LLM is unreliable. We further establish matching lower bounds that characterize the fundamental difficulty of the recourse bandit problem and demonstrate the near-optimality of our algorithms. Experiments on synthetic environments and a real hypertension-management case study confirm that GLRB and LIBRA improve regret, treatment quality, and sample efficiency compared with standard contextual bandits and LLM-only benchmarks. Our results highlight the promise of recourse-aware, LLM-assisted bandit algorithms for trustworthy LLM-bandits collaboration in personalized high-stakes decision-making.
Paper Structure (44 sections, 18 theorems, 116 equations, 11 figures, 2 tables, 4 algorithms)

This paper contains 44 sections, 18 theorems, 116 equations, 11 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Main Contributions and Results
  3. Literature Review
  4. Generalized Linear Recourse Bandit Algorithm
  5. Recourse Optimization Problem
  6. Generalized Linear Recourse Bandit Algorithm (GLRB)
  7. Theoretical Performance Guarantees for the GLRB Algorithm
  8. Language Model Informed Bandit Recourse Algorithm (LIBRA)
  9. The LIBRA Algorithm
  10. Theoretical Performance Guarantees of LIBRA
  11. (i) Benefits from Accurate LLM Guidance.
  12. (ii) Robustness Against Poor LLM Performance.
  13. (iii) Impact and Calibration of the Confidence Threshold Delta.
  14. Lower Bounds on Recourse Regret
  15. Generalized Linear Recourse Bandits with Non-Compliance
  16. ...and 29 more sections

Key Result

Lemma 1

For any given treatment/action $a \in \mathcal{A}$, the optimal recourse derived by solving the recourse optimization problem (eq: RO) is $\check{x}_M^\star=x_M+\gamma\cdot \partial \left\lVert\theta^\star_{a, M}\right\rVert_\star$, where $\partial f$ is a subgradient of the function $f$, and $\left

Figures (11)

  • Figure 1: Illustration of Recourse Bandit (GLRB) and LIBRA frameworks. GLRB offers algorithmic recourses to patients to improve their health conditions for more effective treatment (see § \ref{['sec: GLRB']} for the details of the GLRB algorithm). LIBRA selectively consults LLMs based on uncertainty estimates, enabling a data-driven integration of LLM knowledge and AI recommendations (see § \ref{['sec: LIBRA']} for the details of the LIBRA algorithm and their theoretical properties).
  • Figure 2: Structure of the medical assistant (LLM) prompt example used in our case study.
  • Figure 3: Performance evaluation of different algorithms in terms of cumulative recourse regret and number of LLM queries for patients with hypertension from the ACCORD dataset.
  • Figure 4: Recommended Recourses in terms of DietScore and PhyActHours Suggested by LLM and LIBRA.
  • Figure 5: Recourses Under Non-Compliance Behaviors under both random and adversarial settings.
  • ...and 6 more figures

Theorems & Definitions (22)

  • Example 1: Managing Type 2 Diabetes
  • Lemma 1: Optimal Recourse Under Full Information
  • Corollary 1
  • Lemma 2: High Probability Uncertainty Set
  • Proposition 1: Global Optimality of (\ref{['eq::ORO-Arm']})
  • Lemma 3: Adapted from Corollary 2 in grippo2000convergence
  • Definition 1: Kurdyka--Łojasiewicz (KL) Property
  • Theorem 1: Theoretical Convergence Rate of Algorithm \ref{['alg: two-blockCD']}
  • Definition 2: Recourse Regret
  • Theorem 2: Recourse Regret of the GLRB Algorithm
  • ...and 12 more