Multitask Learning and Bandits via Robust Statistics

TL;DR

This work proposes a novel two-stage multitask learning estimator that exploits this structure in a sample-efficient way, using a unique combination of robust statistics and LASSO regression and illustrates the value of this approach on synthetic and real data sets.

Abstract

Decision-makers often simultaneously face many related but heterogeneous learning problems. For instance, a large retailer may wish to learn product demand at different stores to solve pricing or inventory problems, making it desirable to learn jointly for stores serving similar customers; alternatively, a hospital network may wish to learn patient risk at different providers to allocate personalized interventions, making it desirable to learn jointly for hospitals serving similar patient populations. Motivated by real datasets, we study a natural setting where the unknown parameter in each learning instance can be decomposed into a shared global parameter plus a sparse instance-specific term. We propose a novel two-stage multitask learning estimator that exploits this structure in a sample-efficient way, using a unique combination of robust statistics (to learn across similar instances) and LASSO regression (to debias the results). Our estimator yields improved sample complexity bounds in the feature dimension $d$ relative to commonly-employed estimators; this improvement is exponential for "data-poor" instances, which benefit the most from multitask learning. We illustrate the utility of these results for online learning by embedding our multitask estimator within simultaneous contextual bandit algorithms. We specify a dynamic calibration of our estimator to appropriately balance the bias-variance tradeoff over time, improving the resulting regret bounds in the context dimension $d$. Finally, we illustrate the value of our approach on synthetic and real datasets.

Figures (14)

• Figure 1: Out-of-sample performance (measured by AUC) of a predictive model trained on data from Hospital 1 evaluated on patients from Hospital 1 (green) and Hospitals 2 - 13 (yellow). Point estimates and 95% confidence intervals are based on 1,000 random draws. We observe a significant degradation in predictive performance in non-target hospitals due to dataset shift.
• Figure 2: Heatmap of nonzero coefficients given by the estimated task-specific parameters $\{|\delta^j|\}_{j=1}^{13}$ for each hospital. Each row represents one of 13 hospitals and each column represents one of 77 features extracted from the data. Nonzero coefficients are determined by a bootstrap hypothesis test across 500 random draws of the data (see Appendix \ref{['app:sprs_heatmap']}); we set coefficient $i$ of row $j$ to be zero if the null hypothesis ($\delta_{(i)}^j=0$) is not rejected at a 5% significance level. Each $\|\widehat{\delta}^j\|_0 \ll d$, lending support for our hypothesis of sparse heterogeneity.
• Figure 3: Illustration of Step 1 of our robust multitask estimator for debiasing data collected from multiple instances. Blue squares depict the support; the shade of blue depicts the magnitude. $\mathcal{I}_{\text{poor}}$ represents the index set which can be debiased using the trimmed mean across instances, while $\mathcal{I}_{\text{well}}$ represents the index set which can be debiased using a subsequent LASSO regression for the target instance.
• Figure 4: Bars depict prediction error of one task averaged over 20 trials, with corresponding 95% confidence intervals. 'GLASSO' is group LASSO, 'Nuclear' is nuclear-norm regularization, and 'RME' is RMEstimator.
• Figure 5: Bars depict out-of-sample performance measured by AUC at one hospital (averaged over 1,000 trials), with 95% confidence intervals. Hospitals A and B have 301 and 246 unique patients respectively. 'GLASSO' refers to group LASSO, 'Nuclear' nuclear-norm regularization, and 'RME' our robust multitask estimator.

Theorems & Definitions (98)

• Example 1: Medical Risk Scoring
• Example 2: Demand Prediction
• Definition 1
• Remark 1
• Remark 2
• Definition 2: Well- and poorly-aligned components
• Proposition 4
• Proposition 5
• Theorem 1
• Remark 3