Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation

Naeem Shahabi Sani, Ferial Najiantabriz, Shayan Shafaei, Dean F. Hougen

TL;DR

Three-Channel Evolved Activations is proposed, which is evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take a missingness indicator m and an imputation confidence score c, and results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.

Abstract

Learning in the presence of missing data can result in biased predictions and poor generalizability, among other difficulties, which data imputation methods only partially address. In neural networks, activation functions significantly affect performance yet typical options (e.g., ReLU, Swish) operate only on feature values and do not account for missingness indicators or confidence scores. We propose Three-Channel Evolved Activations (3C-EA), which we evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take (i) the feature value x, (ii) a missingness indicator m, and (iii) an imputation confidence score c. To make these activations useful beyond the input layer, we introduce ChannelProp, an algorithm that deterministically propagates missingness and confidence values via linear layers based on weight magnitudes, retaining reliability signals throughout the network. We evaluate 3C-EA and ChannelProp on datasets with natural and injected (MCAR/MAR/MNAR) missingness at multiple rates under identical preprocessing and splits. Results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.

Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation

TL;DR

Three-Channel Evolved Activations is proposed, which is evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take a missingness indicator m and an imputation confidence score c, and results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.

Abstract

Learning in the presence of missing data can result in biased predictions and poor generalizability, among other difficulties, which data imputation methods only partially address. In neural networks, activation functions significantly affect performance yet typical options (e.g., ReLU, Swish) operate only on feature values and do not account for missingness indicators or confidence scores. We propose Three-Channel Evolved Activations (3C-EA), which we evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take (i) the feature value x, (ii) a missingness indicator m, and (iii) an imputation confidence score c. To make these activations useful beyond the input layer, we introduce ChannelProp, an algorithm that deterministically propagates missingness and confidence values via linear layers based on weight magnitudes, retaining reliability signals throughout the network. We evaluate 3C-EA and ChannelProp on datasets with natural and injected (MCAR/MAR/MNAR) missingness at multiple rates under identical preprocessing and splits. Results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.
Paper Structure (28 sections, 6 equations, 4 figures, 10 tables, 1 algorithm)

This paper contains 28 sections, 6 equations, 4 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Missing Data Handling
  4. Evolved Neural Activation Functions
  5. Method
  6. Problem setup and three-channel inputs
  7. Channel Construction: Imputation, Missingness, and Confidence
  8. ChannelProp: Deterministic Propagation of Reliability Metadata
  9. Fitness evaluation
  10. Evaluation protocol.
  11. Fitness function.
  12. GP Representation and Search
  13. Terminal and function sets.
  14. Evolutionary search.
  15. Experiments, Results, and Discussion
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Tree representation of an evolved three-channel activation function.
  • Figure 2: Performance comparison across missing data rates (10%-50%) on the PIMA dataset.
  • Figure 3: Evolved activation function on heart disease dataset demonstrating three-channel confidence effect.
  • Figure 4: Evolved missing-aware activation functions (Glass dataset).