Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Network Machine Regression (NNMR): A Deep Learning Framework for Uncovering High-order Synergistic Effects

Jiuchen Zhang, Ling Zhou, Peter Song

TL;DR

NNMR integrates a trainable input gating vector $\boldsymbol{\alpha}$ and adaptive depth regularization into a ReLU neural network to jointly perform variable selection and nonlinear function estimation in high dimensions. It couples end-to-end learning with a split-sample permutation-based inference pipeline to rigorously control type I error after selection. Theoretical results provide minimax-optimal risk bounds and selection consistency under mild conditions, while simulations and a real-data application (ELEMENT Mexico City cohort) demonstrate superior variable-selection accuracy, competitive predictive performance, and interpretable sparse architectures compared with BKMR and post-hoc attribution methods. The approach is scalable, GPU-friendly, and yields biologically meaningful predictors, offering a practical framework for high-dimensional nonlinear inference in biomedical and environmental sciences.

Abstract

We propose a new neural network framework, termed Neural Network Machine Regression (NNMR), which integrates trainable input gating and adaptive depth regularization to jointly perform feature selection and function estimation in an end-to-end manner. By penalizing both gating parameters and redundant layers, NNMR yields sparse and interpretable architectures while capturing complex nonlinear relationships driven by high-order synergistic effects. We further develop a post-selection inference procedure based on split-sample, permutation-based hypothesis testing, enabling valid inference without restrictive parametric assumptions. Compared with existing methods, including Bayesian kernel machine regression and widely used post hoc attribution techniques, NNMR scales efficiently to high-dimensional feature spaces while rigorously controlling type I error. Simulation studies demonstrate its superior selection accuracy and inference reliability. Finally, an empirical application reveals sparse, biologically meaningful food group predictors associated with somatic growth among adolescents living in Mexico City.

Neural Network Machine Regression (NNMR): A Deep Learning Framework for Uncovering High-order Synergistic Effects

TL;DR

NNMR integrates a trainable input gating vector and adaptive depth regularization into a ReLU neural network to jointly perform variable selection and nonlinear function estimation in high dimensions. It couples end-to-end learning with a split-sample permutation-based inference pipeline to rigorously control type I error after selection. Theoretical results provide minimax-optimal risk bounds and selection consistency under mild conditions, while simulations and a real-data application (ELEMENT Mexico City cohort) demonstrate superior variable-selection accuracy, competitive predictive performance, and interpretable sparse architectures compared with BKMR and post-hoc attribution methods. The approach is scalable, GPU-friendly, and yields biologically meaningful predictors, offering a practical framework for high-dimensional nonlinear inference in biomedical and environmental sciences.

Abstract

We propose a new neural network framework, termed Neural Network Machine Regression (NNMR), which integrates trainable input gating and adaptive depth regularization to jointly perform feature selection and function estimation in an end-to-end manner. By penalizing both gating parameters and redundant layers, NNMR yields sparse and interpretable architectures while capturing complex nonlinear relationships driven by high-order synergistic effects. We further develop a post-selection inference procedure based on split-sample, permutation-based hypothesis testing, enabling valid inference without restrictive parametric assumptions. Compared with existing methods, including Bayesian kernel machine regression and widely used post hoc attribution techniques, NNMR scales efficiently to high-dimensional feature spaces while rigorously controlling type I error. Simulation studies demonstrate its superior selection accuracy and inference reliability. Finally, an empirical application reveals sparse, biologically meaningful food group predictors associated with somatic growth among adolescents living in Mexico City.
Paper Structure (12 sections, 6 theorems, 63 equations, 2 figures, 3 tables, 2 algorithms)

This paper contains 12 sections, 6 theorems, 63 equations, 2 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Neural Network
  4. Variable Selection
  5. Inference Procedure after Variable Selection
  6. Theory
  7. Computation
  8. Simulations
  9. Consistency of Variable Selection
  10. Post‐selection Inference
  11. Real Data Application
  12. Discussion

Key Result

Theorem 3.1

Suppose the conditions (C1)-(C3) hold. If $\lambda^2_1 \preceq \frac{\log^2 n \log \mathcal{N}_{2n}}{nd} + \frac{\left( \mathcal{R}(g^*_{\mathcal{G}}) - \mathcal{R}(g^*) \right)}{d}$ and $\lambda_2^2 \preceq \frac{\log^2 n \,\log \mathcal{N}_{2n}}{n\mathcal{W}^2\mathcal{D}} \;+\;\frac{(\mathcal{R} where $\mathbb{E}$ is taken with respect to $(\bm{X}_i, Y_i)_{i=1}^n$.

Figures (2)

  • Figure 1: Simple Multi-layer Neural Network
  • Figure 2: Venn diagram of dietary exposures selected by NNMR, SHAP, and DeepLIFT in the ELEMENT study. Displayed are variables with selection frequency exceeding 20% across 100 replicates. Variables in bold indicate those that remained statistically significant after applying the post-selection inference procedure.

Theorems & Definitions (6)

  • Theorem 3.1
  • Corollary 3.1
  • Corollary 3.2
  • Lemma 7.1
  • Lemma 7.2
  • Lemma 7.3