Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Dimensions of a Variational Neuron

Yves Ruffenach

Abstract

We introduce EVE (Elemental Variational Expanse), a variational distributional neuron formulated as a local probabilistic computational unit with an explicit prior, an amortized posterior, and unit-level variational regularization. In most modern architectures, uncertainty is modeled through global latent variables or parameter uncertainty, while the computational unit itself remains scalar. EVE instead relocates probabilistic structure to the neuron level, making it locally observable and controllable. In this paper, the term dimensions refers primarily to the neuron's internal latent dimensionality, denoted by k. We study how varying k, from the atomic case k = 1 to higher-dimensional latent spaces, changes the neuron's learned operating regime. We then examine how this main axis interacts with two additional structural properties: local capacity control and temporal persistence through a neuron-level autoregressive extension. To support this study, EVE is instrumented with internal diagnostics and constraints, including effective KL, a target band on mu^2, out-of-band fractions, and indicators of drift and collapse. Across selected forecasting and tabular settings, we show that latent dimensionality, control, and temporal extension shape the neuron's internal regime, and that some neuron-level variables are measurable, informative, and related to downstream behavior. Overall, the paper provides an experimentally grounded first map of the design space opened by a variational neuron.

Exploring the Dimensions of a Variational Neuron

Abstract

We introduce EVE (Elemental Variational Expanse), a variational distributional neuron formulated as a local probabilistic computational unit with an explicit prior, an amortized posterior, and unit-level variational regularization. In most modern architectures, uncertainty is modeled through global latent variables or parameter uncertainty, while the computational unit itself remains scalar. EVE instead relocates probabilistic structure to the neuron level, making it locally observable and controllable. In this paper, the term dimensions refers primarily to the neuron's internal latent dimensionality, denoted by k. We study how varying k, from the atomic case k = 1 to higher-dimensional latent spaces, changes the neuron's learned operating regime. We then examine how this main axis interacts with two additional structural properties: local capacity control and temporal persistence through a neuron-level autoregressive extension. To support this study, EVE is instrumented with internal diagnostics and constraints, including effective KL, a target band on mu^2, out-of-band fractions, and indicators of drift and collapse. Across selected forecasting and tabular settings, we show that latent dimensionality, control, and temporal extension shape the neuron's internal regime, and that some neuron-level variables are measurable, informative, and related to downstream behavior. Overall, the paper provides an experimentally grounded first map of the design space opened by a variational neuron.
Paper Structure (30 sections, 17 equations, 2 figures, 3 tables)

This paper contains 30 sections, 17 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Positioning logic.
  5. Dimension.
  6. Context.
  7. Time.
  8. Summary.
  9. Method
  10. Principle.
  11. Dimension: a local variational neuron
  12. Aggregation and readout.
  13. Context: local capacity and constraints
  14. Time: local autoregressive dynamics
  15. Unified view and optimization
  16. ...and 15 more sections

Figures (2)

  • Figure 1: Three analytical axes of EVE.A: local latent dimension $k$, from the atomic case $k=1$ to higher-dimensional latent spaces. B: local capacity / operating regime, shaped by KL regularization and capacity control through a target band on $\bar{\mu}^{\,2}$. C: local autoregression in latent space, introducing temporal persistence at the neuron level. Together, these axes define the design space studied.
  • Figure 2: ECL compact-dimension search. Left: KL versus latent dimensionality $k$. Middle: best validation MSE versus $k$. Right: operating-regime map with KL on the horizontal axis, best validation MSE on the vertical axis and color indicating frac high. The compact case $k=1$ yields the most favorable regime under the shared-budget comparison.