Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CHNNet: An Artificial Neural Network With Connected Hidden Neurons

Rafiad Sadat Shahir, Zayed Humayun, Mashrufa Akter Tamim, Shouri Saha, Md. Golam Rabiul Alam, Abu Mohammad Khan

TL;DR

CHNNet addresses the limitation that conventional FNNs lack intra-layer hidden-neuron connections, which restrict lateral information integration. It introduces a forward model with $z^{[l]} = W_1^{[l]} a^{[l-1]} + W_2^{[l]} h^{[l]} + b^{[l]}$, where $h^{[l]} = f_I(W_1^{[l]} a^{[l-1]} + b^{[l]})$ and $a^{[l]} = f^{[l]}(z^{[l]})$, enabling intra-layer mixing. The paper provides a convergence analysis showing faster convergence than standard FNNs and confirms this with empirical results on MNIST and CIFAR-10 for both MLP and CNN backbones. The findings suggest that richer intra-layer connectivity can improve learning efficiency and performance, motivating broader exploration of connectivity patterns beyond depth or width alone.

Abstract

In contrast to biological neural circuits, conventional artificial neural networks are commonly organized as strictly hierarchical architectures that exclude direct connections among neurons within the same layer. Consequently, information flow is primarily confined to feedforward and feedback pathways across layers, which limits lateral interactions and constrains the potential for intra-layer information integration. We introduce an artificial neural network featuring intra-layer connections among hidden neurons to overcome this limitation. Owing to the proposed method for facilitating intra-layer connections, the model is theoretically anticipated to achieve faster convergence compared to conventional feedforward neural networks. The experimental findings provide further validation of the theoretical analysis.

CHNNet: An Artificial Neural Network With Connected Hidden Neurons

TL;DR

CHNNet addresses the limitation that conventional FNNs lack intra-layer hidden-neuron connections, which restrict lateral information integration. It introduces a forward model with , where and , enabling intra-layer mixing. The paper provides a convergence analysis showing faster convergence than standard FNNs and confirms this with empirical results on MNIST and CIFAR-10 for both MLP and CNN backbones. The findings suggest that richer intra-layer connectivity can improve learning efficiency and performance, motivating broader exploration of connectivity patterns beyond depth or width alone.

Abstract

In contrast to biological neural circuits, conventional artificial neural networks are commonly organized as strictly hierarchical architectures that exclude direct connections among neurons within the same layer. Consequently, information flow is primarily confined to feedforward and feedback pathways across layers, which limits lateral interactions and constrains the potential for intra-layer information integration. We introduce an artificial neural network featuring intra-layer connections among hidden neurons to overcome this limitation. Owing to the proposed method for facilitating intra-layer connections, the model is theoretically anticipated to achieve faster convergence compared to conventional feedforward neural networks. The experimental findings provide further validation of the theoretical analysis.
Paper Structure (17 sections, 21 equations, 2 figures, 2 tables)

This paper contains 17 sections, 21 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Theoretical Study
  3. Architecture
  4. Inference Mechanism
  5. Comparison with Multilayer Perceptrons
  6. Comparison with Recurrent Neural Networks
  7. Comparison with Skip-Connections
  8. Learning Mechanism
  9. Convergence Analysis
  10. Empirical Study
  11. Experiment with MLP Architectures
  12. Experiment with CNN Architectures
  13. Convolutional Adaptation of CHNNet
  14. Experiments
  15. Discussion
  16. ...and 2 more sections

Figures (2)

  • Figure 1: The architecture of CHNNet with (a) one hidden layer and (b) two hidden layers.
  • Figure 2: Loss curves on (a–b) MNIST and (c–d) CIFAR‑10 datasets. (a) MLP and CHNNet with unequal parameters, (b) MLP and CHNNet with equal parameters, (c) CNN and CHNNet with unequal parameters, (d) CNN and CHNNet with equal parameters.