Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Bi-Level Variable Selection of Conditional Main Effects for Generalized Linear Models

Kexin Xie, Xinwei Deng

Abstract

Understanding interaction effects among variables is important for regression modeling in various applications. The conventional approach of quantifying interactions as the product of variables often lacks clear interpretability, especially in complex systems. The concept of conditional main effects (CME) provides a more intuitive and interpretable framework for capturing interaction effects by quantifying the effect of one variable conditional on the level of another. A recent method called cmenet further considered the bi-level selection of CMEs by leveraging their natural grouping structure (e.g., sibling and cousin groups) through penalization. However, there are several limitations in the cmenet method, including the coupling ability of penalties for within-group CMEs, lack of adaptiveness for between-group penalties, and restriction to linear models with continuous responses. To overcome these limitations, we propose an adaptive cmenet method for CME selection under the generalized linear model (GLM) framework. The proposed method considers a penalized likelihood approach with adaptive weights to enable effective bi-level variable selection, improving both between-group and within-group selection. An efficient algorithm for parameter estimation is also developed by employing an iteratively reweighted least squares procedure. The performance of the proposed method is evaluated by both simulation studies and real-data studies in gene association analysis.

Adaptive Bi-Level Variable Selection of Conditional Main Effects for Generalized Linear Models

Abstract

Understanding interaction effects among variables is important for regression modeling in various applications. The conventional approach of quantifying interactions as the product of variables often lacks clear interpretability, especially in complex systems. The concept of conditional main effects (CME) provides a more intuitive and interpretable framework for capturing interaction effects by quantifying the effect of one variable conditional on the level of another. A recent method called cmenet further considered the bi-level selection of CMEs by leveraging their natural grouping structure (e.g., sibling and cousin groups) through penalization. However, there are several limitations in the cmenet method, including the coupling ability of penalties for within-group CMEs, lack of adaptiveness for between-group penalties, and restriction to linear models with continuous responses. To overcome these limitations, we propose an adaptive cmenet method for CME selection under the generalized linear model (GLM) framework. The proposed method considers a penalized likelihood approach with adaptive weights to enable effective bi-level variable selection, improving both between-group and within-group selection. An efficient algorithm for parameter estimation is also developed by employing an iteratively reweighted least squares procedure. The performance of the proposed method is evaluated by both simulation studies and real-data studies in gene association analysis.
Paper Structure (17 sections, 5 theorems, 28 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 5 theorems, 28 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Review of Conditional Main Effects
  4. Real-World Motivation
  5. The Proposed Method: Adaptive cmenet
  6. Coupling Ability in the cmenet
  7. The Proposed Adaptive cmenet and glmcmenet
  8. Coupling Ability of the Adaptive cmenet
  9. Parameter Estimation
  10. Convexity and Stability
  11. Threshold Operators
  12. The glmcmenet Algorithm and Parameter Tuning
  13. Simulation Study
  14. Evaluation on adaptive weights
  15. Evaluation on Complex Effect Structures
  16. ...and 2 more sections

Key Result

Proposition 1

Define $\omega_{\max}=\max_{1\leq j \leq p'}\omega_j$ as the maximum individual weight. The objective function $Q(\boldsymbol{\beta})$ in equ7 is strictly convex whenever $\tau + \frac{1}{\gamma \omega_{\max}} < \frac{\zeta_{\min}(\mathbf{X^T WX})}{2n\omega^2_{\max}}$, where $\mathbf{W}$ is evaluate

Figures (5)

  • Figure 1: Effect of varying CME coefficients on selection thresholds of cmenet: (a) Vary $\beta_{A|B+}$; (b) Vary $\beta_{B|A+}$. Curves show the threshold for $\beta_{A|C+}$ (dotted line) and $\beta_{A}$ (solid line) under four contexts: when no other variable included (blue), one sibling included (green), one cousin included (purple), and one sibling and one cousin included (red) in the model. Baseline cmenet setting: $(\lambda_s,\lambda_c,\gamma,\tau)=(1, 0.5, 3, 0.25)$. The adding variable has the same coefficient as $\beta=2$.
  • Figure 2: Effect of varying CME coefficients on selection thresholds of adaptive cmenet: (a) Vary $\beta_{A|B+}$; (b) Vary $\beta_{B|A+}$. Curves show the threshold for $\beta_{A|C+}$ (dotted line) and $\beta_{A}$ (solid line) under four contexts: when no other variable included (blue), one sibling included (green), one cousin included (purple), and one sibling and one cousin included (red) in the model. Baseline adaptive cmenet setting: $(\lambda_s,\lambda_c,\gamma,\tau)=(1, 0.5, 3, 0.25)$. The adding variable has the same coefficient as $\beta=0.5$.
  • Figure 3: Comparison of the threshold function $S(\cdot; \Delta_1, \Delta_2)$ between cmenet and adaptive cmenet across three scenarios: panel (A) No variable included, panel (B) inclusion of one sibling with coefficient $\beta=2$, and panel (C) inclusion of one cousin with coefficient $\beta=2$. Each plot illustrates four threshold functions: (solid) unweighted penalty, i.e., baseline cmenet with settings $(\lambda_s, \lambda_c, \gamma, \tau) = (1, 0.5, 3, 0.25)$; (dash) group-weighted penalty with adaptive weights $(\Omega_{\mathcal{S}(j)}, \Omega_{\mathcal{C}(j)}, \omega_j) = (2/3, 1, 1)$; (long dash) individual-weighted penalty with adaptive weights $(\Omega_{\mathcal{S}(j)}, \Omega_{\mathcal{C}(j)}, \omega_j) = (1, 1, 1.5)$; and (dot dash) combined group and individual-weighted penalty with adaptive weights $(\Omega_{\mathcal{S}(j)}, \Omega_{\mathcal{C}(j)}, \omega_j) = (2/3, 1, 1.5)$.
  • Figure 4: Correlation matrix of the simulated full model matrix ($n=100$, $p=3$) generated from the equicorrelation latent model with (A) $\rho=0$ and (B) $\rho=1/\sqrt{2}$.
  • Figure 5: Example 2. Comparison to hierNet with $n=50$, $p=20$ for (A) continuous response with uncorrelated MEs $\rho=0$, (B) continuous response with correlated MEs $\rho=1/\sqrt{2}$, (C) binary response with uncorrelated MEs $\rho=0$, and (D) binary response with correlated MEs $\rho=1/\sqrt{2}$. The x-axis shows the number of active groups, while the y-axis displays the total # of selected effects, and MSPE (for continuous) or Misclassification Rate (for binary). Each column corresponds to a type of active effect: main+cousins (left), and main+siblings (right).

Theorems & Definitions (7)

  • Example 1
  • Proposition 1
  • Corollary 1.1
  • Remark 1
  • Lemma 1
  • Theorem 1
  • Proposition 2