Bayesian Controlled FDR Variable Selection via Knockoffs

TL;DR

The paper addresses variable selection under false discovery rate control when covariates are potentially correlated by extending the model-X knockoff filter to a fully Bayesian framework. It builds a joint model where knockoffs are latent and integrated into posterior inference, leveraging a Gaussian graphical model to encode covariate dependencies and a spike-and-slab prior to select active variables, with BFDR control guaranteed via an upper bound on the non-inclusion probability. Key contributions include a valid Bayesian knockoff construction, posterior-based FDR control in finite samples (and asymptotically with estimated covariate structure), and demonstrations that the approach yields more stable selections than classical knockoffs and competitive performance versus other Bayesian selectors. The method is validated through extensive simulations (including dependent covariates and survival outcomes) and an application to prostate cancer data, illustrating practical utility and improved stability in high-dimensional settings.

Abstract

In many research fields, researchers aim to identify significant associations between a set of explanatory variables and a response while controlling the false discovery rate (FDR). The Knockoff filter has been recently proposed in the frequentist paradigm to introduce controlled noise in a model by cleverly constructing copies of the predictors as auxiliary variables. In this paper, we develop a fully Bayesian generalization of the classical model-X knockoff filter for normally distributed covariates. In our approach we consider a joint model of the covariates and the response variables, and incorporate the conditional independence structure of the covariates into the prior distribution of the auxiliary knockoff variables. We further incorporate the estimation of a graphical model among the covariates,leading to improved knockoffs generation and estimation of the covariate effects on the response. We use a modified spike-and-slab prior on the regression coefficients, which avoids the increase of the model dimension as typical in the classical knockoff filter. Our model performs variable selection using an upper bound on the posterior probability of non-inclusion. We show how our construction leads to valid model-X knockoffs and demonstrate that the proposed variable selection procedure leads to controlling the Bayesian FDR at an arbitrary level, in finite samples, if the distribution of the covariates is fully known, and asymptotically if estimated as in the proposed model. We use simulated data to demonstrate that our proposal increases the stability of the selection with respect to classical knockoff methods. With respect to Bayesian variable selection methods, we show that our selection procedure achieves comparable or better performances, while maintaining control over the FDR. Finally, we show the usefulness of the proposed model with an application to real data.

Figures (4)

• Figure 1: Simulation study: Scale-free graph structure under Gaussian graphical model in Scenario $2$. White nodes correspond to the noise variables, and blue nodes to the active ones. A stronger blue color corresponds to a larger effect.
• Figure 2: Simulation study: Variable selection performed by our Bayesian knockoff filter method (BayeKnock) and by spike-and-regression (Spike-Slab) on a simulated dataset under Scenario 3. Top plot (BayesKnock): Bars represent estimates of the upper bound for ${\mathbb{P}}[r_j=0\mid D]$ as in Equation \ref{['ubound']} while points represent the estimated BFDR. Red points correspond to true active variables. Bottom plot (Spike-Slab): Estimated marginal posterior probabilities of inclusion (PPIs). Red bars correspond to true active variables.
• Figure 3: Prostate cancer data: Posterior distribution of $W_j$, $j=1, \dots, 6$. Those highlighted in blue are selected according to Figure \ref{['fig:selection']}
• Figure 4: Prostate cancer data: Variable selection. Bars represent estimates $2\, \widehat{\mathbb{P}}[W_j \le 0\mid \bm D]$, $j = 1, \ldots 6$, in increasing order and points represent the estimated $\text{BFDR}(\mathcal{S})$ with $\mathcal{S}$ the set of all previous indexes. The dashed red line is the chosen threshold $q=0.1$.

Theorems & Definitions (6)

• Proposition 1
• proof
• Proposition 2
• proof
• Proposition 3
• proof