Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-dimensional Bayesian Tobit regression for censored response with Horseshoe prior

The Tien Mai

TL;DR

This work tackles high-dimensional regression with left-censored responses by introducing a Bayesian sparse Tobit model that uses a Horseshoe prior and a fractional posterior to enforce sparsity and obtain reliable inference. A data-augmentation–based Gibbs sampler enables efficient posterior computation, and the authors prove posterior concentration results showing that the fractional posterior contracts at near-minimax rates under sparsity. Empirically, the method consistently outperforms a recent Lasso–Tobit approach across simulations with varying sparsity, correlation, and model misspecification, and it yields improved predictive accuracy on a censored gene-expression dataset. The combination of theoretical guarantees, scalable computation, and strong empirical performance makes this a valuable tool for censored, high-dimensional data analysis, with an accompanying R package for practical use.

Abstract

Censored response variables--where outcomes are only partially observed due to known bounds--arise in numerous scientific domains and present serious challenges for regression analysis. The Tobit model, a classical solution for handling left-censoring, has been widely used in economics and beyond. However, with the increasing prevalence of high-dimensional data, where the number of covariates exceeds the sample size, traditional Tobit methods become inadequate. While frequentist approaches for high-dimensional Tobit regression have recently been developed, notably through Lasso-based estimators, the Bayesian literature remains sparse and lacks theoretical guarantees. In this work, we propose a novel Bayesian framework for high-dimensional Tobit regression that addresses both censoring and sparsity. Our method leverages the Horseshoe prior to induce shrinkage and employs a data augmentation strategy to facilitate efficient posterior computation via Gibbs sampling. We establish posterior consistency and derive concentration rates under sparsity, providing the first theoretical results for Bayesian Tobit models in high dimensions. Numerical experiments show that our approach outperforms favorably with the recent Lasso-Tobit method. Our method is implemented in the R package tobitbayes, which can be found on Github.

High-dimensional Bayesian Tobit regression for censored response with Horseshoe prior

TL;DR

This work tackles high-dimensional regression with left-censored responses by introducing a Bayesian sparse Tobit model that uses a Horseshoe prior and a fractional posterior to enforce sparsity and obtain reliable inference. A data-augmentation–based Gibbs sampler enables efficient posterior computation, and the authors prove posterior concentration results showing that the fractional posterior contracts at near-minimax rates under sparsity. Empirically, the method consistently outperforms a recent Lasso–Tobit approach across simulations with varying sparsity, correlation, and model misspecification, and it yields improved predictive accuracy on a censored gene-expression dataset. The combination of theoretical guarantees, scalable computation, and strong empirical performance makes this a valuable tool for censored, high-dimensional data analysis, with an accompanying R package for practical use.

Abstract

Censored response variables--where outcomes are only partially observed due to known bounds--arise in numerous scientific domains and present serious challenges for regression analysis. The Tobit model, a classical solution for handling left-censoring, has been widely used in economics and beyond. However, with the increasing prevalence of high-dimensional data, where the number of covariates exceeds the sample size, traditional Tobit methods become inadequate. While frequentist approaches for high-dimensional Tobit regression have recently been developed, notably through Lasso-based estimators, the Bayesian literature remains sparse and lacks theoretical guarantees. In this work, we propose a novel Bayesian framework for high-dimensional Tobit regression that addresses both censoring and sparsity. Our method leverages the Horseshoe prior to induce shrinkage and employs a data augmentation strategy to facilitate efficient posterior computation via Gibbs sampling. We establish posterior consistency and derive concentration rates under sparsity, providing the first theoretical results for Bayesian Tobit models in high dimensions. Numerical experiments show that our approach outperforms favorably with the recent Lasso-Tobit method. Our method is implemented in the R package tobitbayes, which can be found on Github.
Paper Structure (17 sections, 5 theorems, 37 equations, 3 figures, 6 tables)

This paper contains 17 sections, 5 theorems, 37 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Bayesian sparse Tobit regression
  3. Sparse Tobit model for censored response
  4. Method
  5. Theoretical result
  6. A Gibbs sampling implementation
  7. Numerical studies
  8. Setup
  9. Simulation results
  10. Results with large sample sizes
  11. Results with model violation
  12. Results on running times
  13. Application: censored prediction using gene expression data
  14. Conclusion
  15. Proofs
  16. ...and 2 more sections

Key Result

Theorem 1

For any $\alpha\in(0,1)$, assume that Assumption assum_grow_p, assum_beta0_bounded and asmum_lip_alquier hold. We have that where $\varepsilon_n = K s^* \log \left( p /s^*\right) / n$, for some numerical constant $K>0$ depending only on $C_1, B_1, B_2$.

Figures (3)

  • Figure 1: Log-log plot of computation times. Left: Varying sample size with fixed $p = 100, s^* = 10$. Right: Varying number of predictors with fixed $n = 100, s^* = 10$.
  • Figure 2: Trace plots from the Gibbs sampler for selected parameter entries. Top row: three randomly chosen entries with true value 1. Middle row: three randomly chosen entries with true value $-1$. Bottom row: three randomly chosen entries with true value 0. Red lines show the true values.
  • Figure 3: ACF plots from the Gibbs sampler for some random entries as in Figure \ref{['fig_tracplot']}. Top row (3 plots): 3 random entries with true value 1. Middle row (3 plots): 3 random entries with true value $-1$. Bottom row (3 plots): 3 random entries with true value 0.

Theorems & Definitions (7)

  • Theorem 1
  • Corollary 2.1
  • Remark 1
  • proof : Proof of Theorem \ref{['theorem_result_dis_expectation']}
  • Theorem 2: Theorem 2.6 in alquier2020concentration
  • Theorem 3: Corollary 2.5 in alquier2020concentration
  • Lemma 1: Lemma 3 in mai2024concentration