GRIP2: A Robust and Powerful Deep Knockoff Method for Feature Selection

TL;DR

GRIP2 tackles robust feature selection under finite-sample $FDR$ control in difficult regimes of high feature correlation and low $SNR$ by introducing two-dimensional persistence of first-layer group activity across a regularization surface parameterized by sparsity $\lambda$ and geometry $a$. A Block Stochastic Sampling procedure enables single-run estimation of surface-averaged activity, producing antisymmetric statistics $W_j$ that are compatible with Model-X knockoffs. Empirical results on synthetic, semi-real, and real HIV data show GRIP2 improves power and stability while maintaining valid $FDR$ control, outperforming several nonlinear and linear baselines. The work demonstrates practical utility for reliable discoveries in settings with strong feature redundancy and complex nonlinear relationships, such as genomic mutations and drug resistance mutations.

Abstract

Identifying truly predictive covariates while strictly controlling false discoveries remains a fundamental challenge in nonlinear, highly correlated, and low signal-to-noise regimes, where deep learning based feature selection methods are most attractive. We propose Group Regularization Importance Persistence in 2 Dimensions (GRIP2), a deep knockoff feature importance statistic that integrates first-layer feature activity over a two-dimensional regularization surface controlling both sparsity strength and sparsification geometry. To approximate this surface integral in a single training run, we introduce efficient block-stochastic sampling, which aggregates feature activity magnitudes across diverse regularization regimes along the optimization trajectory. The resulting statistics are antisymmetric by construction, ensuring finite-sample FDR control. In extensive experiments on synthetic and semi-real data, GRIP2 demonstrates improved robustness to feature correlation and noise level: in high correlation and low signal-to-noise ratio regimes where standard deep learning based feature selectors may struggle, our method retains high power and stability. Finally, on real-world HIV drug resistance data, GRIP2 recovers known resistance-associated mutations with power better than established linear baselines, confirming its reliability in practice.

Figures (7)

• Figure 1: Adversarial high-correlation regime ($\rho=0.8$). Power (top) and selection stability (bottom; average pairwise Jaccard) versus target FDR $q$. Power increases with $q$ while stability typically declines. Compared to other methods, GRIP2 maintains a favorable power--stability trade-off across $q$.
• Figure 2: Robustness to correlation strength. Power (a) and stability (b) as a function of $\rho \in [0,0.8]$ at target FDR $q=0.1$. Competing methods exhibit a degradation as correlation increases, whereas GRIP2 performs better at higher $\rho$, indicating robustness to adversarial correlation through integration over sparsity geometries.
• Figure 3: Effect of seed ensembling ($\rho=0.8$, target FDR $q=0.1$). Power (a) and stability (b) versus ensemble size $K$. Increasing $K$ yields limited gains for most baselines and does not resolve structural instability caused by correlated decoys. GRIP2 achieves strong stability and power with a single training run without requiring seed ensembling
• Figure 4: FDR validity on HAR data. Realized FDR versus target FDR. GRIP2 maintains valid FDR control across all $q$, while DeepPINK and LAPA frequently exceeds the nominal level, indicating invalid FDR control under approximate knockoff construction.
• Figure 5: Power and Stability vs Target FDR on HAR data. GRIP2 consistently achieve the best power while maintaining favorable stability, whereas other methods either only achieves stability with low power or are worse on both metrics