Efficient and Interpretable Neural Networks Using Complex Lehmer Transform

TL;DR

The paper addresses the need for interpretable and efficient neural networks by introducing Lehmer Activation Units based on real-valued and complex-valued weighted Lehmer transforms, enabling adaptive feature aggregation and phase-sensitive interactions. It develops definitions for L(s; x, w) and its complex extension with s = a + bi, analyzes monotonicity and Schur convexity, and demonstrates that a single layer of LAUs can achieve competitive accuracy on Iris, Wine, WBC, and MNIST with reduced architectural complexity. The experiments show real LAUs performing strongly and complex LAUs reaching up to 98% accuracy on MNIST, validating the efficiency and representational power of the approach. Overall, the work provides a theoretically grounded, interpretable activation framework for both real and complex neural networks with broad applicability to time-series, signal processing, and high-dimensional data.

Abstract

We propose an efficient and interpretable neural network with a novel activation function called the weighted Lehmer transform. This new activation function enables adaptive feature selection and extends to the complex domain, capturing phase-sensitive and hierarchical relationships within data. Notably, it provides greater interpretability and transparency compared to existing machine learning models, facilitating a deeper understanding of its functionality and decision-making processes. We analyze the mathematical properties of both real-valued and complex-valued Lehmer activation units and demonstrate their applications in modeling nonlinear interactions. Empirical evaluations demonstrate that our proposed neural network achieves competitive accuracy on benchmark datasets with significantly improved computational efficiency. A single layer of real-valued or complex-valued Lehmer activation units is shown to deliver state-of-the-art performance, balancing efficiency with interpretability.