Triple/Double-Debiased Lasso

Abstract

In this paper, we propose a triple (or double-debiased) Lasso estimator for inference on a low-dimensional parameter in high-dimensional linear regression models. The estimator is based on a moment function that satisfies not only first- but also second-order Neyman orthogonality conditions, thereby eliminating both the leading bias and the second-order bias induced by regularization. We derive an asymptotic linear representation for the proposed estimator and show that its remainder terms are never larger and are often smaller in order than those in the corresponding asymptotic linear representation for the standard double Lasso estimator. Because of this improvement, the triple Lasso estimator often yields more accurate finite-sample inference and confidence intervals with better coverage. Monte Carlo simulations confirm these gains. In addition, we provide a general recursive formula for constructing higher-order Neyman orthogonal moment functions in Z-estimation problems, which underlies the proposed estimator as a special case.

Figures (10)

• Figure 1: Monte Carlo squared bias. Rows correspond to the three sparsity regimes for $\boldsymbol{\gamma}_0$, and columns correspond to $(n,p)\in\{(500,250),(1000,500),(2000,1000)\}$. Triple lasso sharply lowers squared bias in the intermediate and approximate sparsity designs.
• Figure 2: Monte Carlo variance. Rows correspond to the three sparsity regimes for $\boldsymbol{\gamma}_0$, and columns correspond to $(n,p)\in\{(500,250),(1000,500),(2000,1000)\}$. Triple lasso typically incurs a moderate variance increase relative to double lasso.
• Figure 3: Monte Carlo mean squared error. Triple lasso typically improves MSE in the harder designs, especially under intermediate and approximate sparsity.
• Figure 4: Monte Carlo coverage probability for nominal $95\%$ confidence intervals. Triple lasso brings coverage much closer to the nominal level when double lasso undercovers because of residual bias.
• Figure 5: Monte Carlo mean confidence interval length. Triple lasso intervals are somewhat longer, reflecting the additional uncertainty from the second-order correction, but the increase in length is accompanied by substantially more reliable coverage in difficult designs.

Theorems & Definitions (26)

• Lemma 2.1
• Remark 2.1
• Remark 2.2
• Remark 2.3
• Remark 2.4
• Theorem 2.1
• Lemma 3.1
• Remark 3.1
• Remark 3.2
• proof : Proof of Lemma \ref{['lem: novel score']}