Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Scale-Invariant Diagnostic Approach Towards Understanding Dynamics of Deep Neural Networks

Ambarish Moharil, Damian Tamburri, Indika Kumara, Willem-Jan Van Den Heuvel, Alireza Azarfar

TL;DR

The paper tackles intrinsic explainability for deep neural networks by introducing a scale-invariant, fractal-based framework that analyzes nonlinear dynamics across multiple observation scales. It segmentally analyzes weight matrices to compute fractal features via box-counting with $FD=\frac{\ln(N)}{\ln(1/r)}$, applies an exponential kernel to relate fractal segments, and learns a fractal topology with a Graph-Based Neural Network to form a fractal hypergraph. Preliminary MNIST experiments demonstrate phase-flow dynamics with decreasing gradient norms and attractor convergence, suggesting enhanced stability and interpretability of training dynamics. This approach potentially advances XAI by revealing multiscale dynamical structure that underpins decision behavior in DNNs, guiding intrinsic explanations rather than post-hoc surrogates.

Abstract

This paper introduces a scale-invariant methodology employing \textit{Fractal Geometry} to analyze and explain the nonlinear dynamics of complex connectionist systems. By leveraging architectural self-similarity in Deep Neural Networks (DNNs), we quantify fractal dimensions and \textit{roughness} to deeply understand their dynamics and enhance the quality of \textit{intrinsic} explanations. Our approach integrates principles from Chaos Theory to improve visualizations of fractal evolution and utilizes a Graph-Based Neural Network for reconstructing network topology. This strategy aims at advancing the \textit{intrinsic} explainability of connectionist Artificial Intelligence (AI) systems.

A Scale-Invariant Diagnostic Approach Towards Understanding Dynamics of Deep Neural Networks

TL;DR

The paper tackles intrinsic explainability for deep neural networks by introducing a scale-invariant, fractal-based framework that analyzes nonlinear dynamics across multiple observation scales. It segmentally analyzes weight matrices to compute fractal features via box-counting with , applies an exponential kernel to relate fractal segments, and learns a fractal topology with a Graph-Based Neural Network to form a fractal hypergraph. Preliminary MNIST experiments demonstrate phase-flow dynamics with decreasing gradient norms and attractor convergence, suggesting enhanced stability and interpretability of training dynamics. This approach potentially advances XAI by revealing multiscale dynamical structure that underpins decision behavior in DNNs, guiding intrinsic explanations rather than post-hoc surrogates.

Abstract

This paper introduces a scale-invariant methodology employing \textit{Fractal Geometry} to analyze and explain the nonlinear dynamics of complex connectionist systems. By leveraging architectural self-similarity in Deep Neural Networks (DNNs), we quantify fractal dimensions and \textit{roughness} to deeply understand their dynamics and enhance the quality of \textit{intrinsic} explanations. Our approach integrates principles from Chaos Theory to improve visualizations of fractal evolution and utilizes a Graph-Based Neural Network for reconstructing network topology. This strategy aims at advancing the \textit{intrinsic} explainability of connectionist Artificial Intelligence (AI) systems.
Paper Structure (4 sections, 5 equations, 3 figures)

This paper contains 4 sections, 5 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Proposed Methodology
  3. Preliminary Results
  4. Research Roadmap

Figures (3)

  • Figure 1: Visualizing the $\#$segments (log scale) at various valid scales $r$ for different matrix dimensions $n \times m$.
  • Figure 2: Left image shows features learned by neuron-channels across two convolution layers. Right image visualizes the exponential kernel interactions between fractal segment features for various $\gamma$ values.
  • Figure 3: Phase-Flow Diagrams