Calibrating confounding strength in sensitivity models for weighting estimators: a comparative review and a new method

Abstract

Causal inference is only valid when its underlying assumptions are satisfied, one of the most central being the ignorability or unconfoundedness assumption. However, this hypothesis is often unrealistic in observational studies, as some confounding variables may remain unobserved. To address this limitation, sensitivity models for Inverse Probability Weighting (IPW) estimators, known as Marginal Sensitivity Models, have been introduced, allowing for a controlled relaxation of ignorability. A substantial body of literature has emerged around these models, aiming to derive sharp and robust bounds for both binary and continuous treatment effects. A key element of these approaches is the specification of a sensitivity parameter, referred to as the "confounding strength", which quantifies the extent of deviation from ignorability. Yet, determining an appropriate value for this parameter is challenging, and the final interpretation of sensitivity analyses can be unclear. We believe these difficulties represent major obstacles to the adoption of such methods in practice. Therefore, after introducing sensitivity analyses for IPW estimators, we review different strategies to estimate or lower bound the confounding strength, introduce a new method leveraging negative controls, provide a decision tree with guidelines to choose a suitable approach, and compare the methodologies in an in-depth simulation study.

Figures (21)

• Figure 1: Illustration of the quantities involved in the sensitivity analysis for $\theta(1)$ (in dark blue), as defined by Equation \ref{['eqn:true_ipw_theta_1']}. $\hat{\theta}(1)$ (in dark red), as defined by Equation \ref{['eqn:conf_estim_theta_1']}, is the estimation of $\theta(1)$ under $\mathbf{X}$-ignorability. $\theta(1)$ is bounded by $\theta_-(1, \Gamma)$ and $\theta_+(1, \Gamma)$, the Partially Identified set (PI set) (in blue), as defined by Equation \ref{['eqn:theta_1_PIS']}. Estimating the PI set gives the Point Estimate Interval (PEI) $[\hat{\theta}(1, \Gamma)_-, \hat{\theta}(1, \Gamma)_+]$ (in red), as defined by Equation \ref{['eqn:theta_1_PEI']}. Finally, the PEI can be used to get a Confidence Interval (CI) (in green). When $\Gamma = 1$, the PI set reduces to $\theta(1)$ and the PEI reduces to $\hat{\theta}(1)$.
• Figure 2: Different configurations of sensitivity analysis for the ATE. Figure \ref{['fig:sa_critical_value_ATE']}a depicts the case where the critical value $\Gamma_c = 1$ and the confidence intervals (in blue) contain the null effect for all values of $\Gamma$ (no robustness to unobserved confounders). Figure \ref{['fig:sa_critical_value_ATE']}b shows the opposite case where $\Gamma_c = +\infty$ and no confidence interval contains the null effect (robustness to unobserved confounders of any strength). Figure \ref{['fig:sa_critical_value_ATE']}c corresponds to the intermediate case, where $\Gamma_c \in (1, +\infty)$. If the estimated $\hat{\Gamma}$ is lower than $\Gamma_c$, the conclusions are robust to unobserved confounders (blue area), but if it is larger, the conclusions may not be robust to possible unobserved confounders (orange area).
• Figure 3: Different configurations of sensitivity analysis for the APO. The dotted line in light blue is the estimated APO curve. The red dotted line corresponds to the null effect. The critical value $\Gamma_c$ is defined as the lowest value of $\Gamma$ for which all intervals between expositions $t_1$ and $t_2$ contain the null effect (dark green curve). Figure \ref{['fig:sa_critical_value_APO']}a depicts the case where the critical value $\Gamma_c = 1$ and the confidence intervals between $t_1$ and $t_2$ contain the null effect for all values of $\Gamma$ (no robustness to unobserved confounders). Figure \ref{['fig:sa_critical_value_APO']}b shows the opposite case where $\Gamma_c = +\infty$ and the null effect is not in all confidence intervals between $t_1$ and $t_2$ (robustness to unobserved confounders of any strength). Figure \ref{['fig:sa_critical_value_APO']}c corresponds to the intermediate case, where $\Gamma_c \in (1, +\infty)$. If the estimated $\hat{\Gamma}$ is lower than $\Gamma_c$, the conclusions are robust to unobserved confounders (blue curves), but if it is larger, the conclusions may not be robust to possible unobserved confounders (orange curves).
• Figure 4: Three main cases when computing $\hat{\phi}_\diamond(\Gamma, \alpha)$. The green arrow represents the numerator of the test statistics $\hat{T}_\Gamma^-$ and the red arrow, the numerator of the test statistics $\hat{T}_\Gamma^+$. In Figures \ref{['fig:via_rct']}a and \ref{['fig:via_rct']}b, $H_0(\Gamma)$ is not rejected, whereas in Figure \ref{['fig:via_rct']}c, $H_0(\Gamma)$ is rejected. The significance level $\alpha$, and therefore, the quantile $z_{\alpha/2}$ determine what "close to" means in \ref{['fig:via_rct']}b.
• Figure 5: Examples of causal diagrams of a negative control outcome (NCO) $W$. An arrow directed from $A$ to $B$ indicates that $A$ causes $B$. Dashed edges may be absent. An ideal NCO $W$ (a) is such that the set of common causes $\mathbf{U}$ between $T$ and $Y$ and between $T$ and $W$ is the same. An approximately $\mathbf{U}$-comparable NCO $W$ (b, c) is such that an additional unobserved confounder $\mathbf{U}_2$ biases the relation between $T$ and $W$ (case 1) or between $T$ and $Y$ (case 2). In all cases, $T$ does not cause $W$.

Theorems & Definitions (8)

• Proposition 2.4
• Definition 2.5: MSM, formulation with $\mathbf{U}$
• Definition 2.6: CMSM, formulation with $\mathbf{U}$
• Definition A.2: RSM, formulation with $\mathbf{U}$
• Definition A.3: RSM, formulation with $Y(0)$ or $Y(1)$
• Definition A.4: MSM, formulation with $Y(0)$ or $Y(1)$
• Definition A.5: CMSM, formulation with $Y(t)$
• Remark B.1