Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bi-Level Contextual Bandits for Individualized Resource Allocation under Delayed Feedback

Mohammadsina Almasi, Hadis Anahideh

TL;DR

This work develops MetaCUB, a bi-level contextual bandit framework for equitable resource allocation under delayed feedback and dynamic cohorts. The meta level allocates subgroup budgets while the base level selectively targets individuals using a learned outcome model $f(\mathbf{x})$, incorporating resource-specific delay kernels $K^{r}$ and allocation cooldowns $c^{r}$. The approach yields a constraint-aware, delay-aware policy with theoretical fairness guarantees and strong empirical performance on the Educational Longitudinal Study (ELS) and JOBS datasets, achieving lower cumulative regret and near-parity subgroup outcomes. By explicitly modeling cohort dynamics and temporal delays, MetaCUB offers a scalable, deployment-ready solution for policy- and social-welfare-oriented resource distribution.

Abstract

Equitably allocating limited resources in high-stakes domains-such as education, employment, and healthcare-requires balancing short-term utility with long-term impact, while accounting for delayed outcomes, hidden heterogeneity, and ethical constraints. However, most learning-based allocation frameworks either assume immediate feedback or ignore the complex interplay between individual characteristics and intervention dynamics. We propose a novel bi-level contextual bandit framework for individualized resource allocation under delayed feedback, designed to operate in real-world settings with dynamic populations, capacity constraints, and time-sensitive impact. At the meta level, the model optimizes subgroup-level budget allocations to satisfy fairness and operational constraints. At the base level, it identifies the most responsive individuals within each group using a neural network trained on observational data, while respecting cooldown windows and delayed treatment effects modeled via resource-specific delay kernels. By explicitly modeling temporal dynamics and feedback delays, the algorithm continually refines its policy as new data arrive, enabling more responsive and adaptive decision-making. We validate our approach on two real-world datasets from education and workforce development, showing that it achieves higher cumulative outcomes, better adapts to delay structures, and ensures equitable distribution across subgroups. Our results highlight the potential of delay-aware, data-driven decision-making systems to improve institutional policy and social welfare.

Bi-Level Contextual Bandits for Individualized Resource Allocation under Delayed Feedback

TL;DR

This work develops MetaCUB, a bi-level contextual bandit framework for equitable resource allocation under delayed feedback and dynamic cohorts. The meta level allocates subgroup budgets while the base level selectively targets individuals using a learned outcome model , incorporating resource-specific delay kernels and allocation cooldowns . The approach yields a constraint-aware, delay-aware policy with theoretical fairness guarantees and strong empirical performance on the Educational Longitudinal Study (ELS) and JOBS datasets, achieving lower cumulative regret and near-parity subgroup outcomes. By explicitly modeling cohort dynamics and temporal delays, MetaCUB offers a scalable, deployment-ready solution for policy- and social-welfare-oriented resource distribution.

Abstract

Equitably allocating limited resources in high-stakes domains-such as education, employment, and healthcare-requires balancing short-term utility with long-term impact, while accounting for delayed outcomes, hidden heterogeneity, and ethical constraints. However, most learning-based allocation frameworks either assume immediate feedback or ignore the complex interplay between individual characteristics and intervention dynamics. We propose a novel bi-level contextual bandit framework for individualized resource allocation under delayed feedback, designed to operate in real-world settings with dynamic populations, capacity constraints, and time-sensitive impact. At the meta level, the model optimizes subgroup-level budget allocations to satisfy fairness and operational constraints. At the base level, it identifies the most responsive individuals within each group using a neural network trained on observational data, while respecting cooldown windows and delayed treatment effects modeled via resource-specific delay kernels. By explicitly modeling temporal dynamics and feedback delays, the algorithm continually refines its policy as new data arrive, enabling more responsive and adaptive decision-making. We validate our approach on two real-world datasets from education and workforce development, showing that it achieves higher cumulative outcomes, better adapts to delay structures, and ensures equitable distribution across subgroups. Our results highlight the potential of delay-aware, data-driven decision-making systems to improve institutional policy and social welfare.

Paper Structure

This paper contains 15 sections, 1 theorem, 15 equations, 5 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Setup
  4. Proposed Approach
  5. Fairness Properties of MetaCUB
  6. Experiments
  7. Results
  8. Conclusion
  9. Appendix
  10. Constrained Contextual MAB Algorithm
  11. Proof of Lemma\ref{['lemma1']}
  12. Baseline Methods and Characteristics
  13. Datasets
  14. Dataset Specifications Comparison.
  15. Fairness Analysis

Key Result

Lemma 1

Let $\mathcal{A}_{\texttt{MetaCUB}}$ denote the bi-level allocation under MetaCUB, and $\mathcal{A}_{\texttt{Flat}}$ denote the allocation from a one-level contextual bandit (e.g., LinUCB) that selects individual-resource pairs without subgroup constraints. Define disparity as the difference between

Figures (5)

  • Figure 1: Delay kernel distributions.
  • Figure 2: ELS, Delay Kernel Type-I: Cumulative regret.
  • Figure 3: ELS, Delay Kernel Type-II: Cumulative regret.
  • Figure 4: JOBS, Delay Kernel Type-I: Cumulative regret.
  • Figure 5: JOBS, Delay Kernel Type-II: Cumulative regret.

Theorems & Definitions (1)

  • Lemma 1: Disparity Reduction