Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scale-Consistent State-Space Dynamics via Fractal of Stationary Transformations

Geunhyeok Yu, Hyoseok Hwang

TL;DR

The paper tackles the problem that deep models lack geometric guarantees for intermediate representations, complicating adaptive depth and early stopping. It introduces Fractal of Stationary Transformations (FROST), a fractal inductive bias that enforces scale-consistent latent dynamics and self-similar manifolds through stationary, contractive updates governed by a scale factor $\lambda$ and Hurst exponent $H$. The approach enables a ranking-based halting mechanism that selects computation depth based on intrinsic representation quality, with a training objective that combines a task loss and two ranking losses. Theoretical results show contraction to a unique fixed point and exponential error decay, while empirical results on ImageNet-100 and CIFAR-100 demonstrate improved adaptive efficiency, stable training, and higher throughput compared to non-fractal baselines, highlighting the importance of latent geometry in enabling principled anytime predictions.

Abstract

Recent deep learning models increasingly rely on depth without structural guarantees on the validity of intermediate representations, rendering early stopping and adaptive computation ill-posed. We address this limitation by formulating a structural requirement for state-space model's scale-consistent latent dynamics across iterative refinement, and derive Fractal of Stationary Transformations (FROST), which enforces a self-similar representation manifold through a fractal inductive bias. Under this geometry, intermediate states correspond to different resolutions of a shared representation, and we provide a geometric analysis establishing contraction and stable convergence across iterations. As a consequence of this scale-consistent structure, halting naturally admits a ranking-based formulation driven by intrinsic feature quality rather than extrinsic objectives. Controlled experiments on ImageNet-100 empirically verify the predicted scale-consistent behavior, showing that adaptive efficiency emerges from the aligned latent geometry.

Scale-Consistent State-Space Dynamics via Fractal of Stationary Transformations

TL;DR

The paper tackles the problem that deep models lack geometric guarantees for intermediate representations, complicating adaptive depth and early stopping. It introduces Fractal of Stationary Transformations (FROST), a fractal inductive bias that enforces scale-consistent latent dynamics and self-similar manifolds through stationary, contractive updates governed by a scale factor and Hurst exponent . The approach enables a ranking-based halting mechanism that selects computation depth based on intrinsic representation quality, with a training objective that combines a task loss and two ranking losses. Theoretical results show contraction to a unique fixed point and exponential error decay, while empirical results on ImageNet-100 and CIFAR-100 demonstrate improved adaptive efficiency, stable training, and higher throughput compared to non-fractal baselines, highlighting the importance of latent geometry in enabling principled anytime predictions.

Abstract

Recent deep learning models increasingly rely on depth without structural guarantees on the validity of intermediate representations, rendering early stopping and adaptive computation ill-posed. We address this limitation by formulating a structural requirement for state-space model's scale-consistent latent dynamics across iterative refinement, and derive Fractal of Stationary Transformations (FROST), which enforces a self-similar representation manifold through a fractal inductive bias. Under this geometry, intermediate states correspond to different resolutions of a shared representation, and we provide a geometric analysis establishing contraction and stable convergence across iterations. As a consequence of this scale-consistent structure, halting naturally admits a ranking-based formulation driven by intrinsic feature quality rather than extrinsic objectives. Controlled experiments on ImageNet-100 empirically verify the predicted scale-consistent behavior, showing that adaptive efficiency emerges from the aligned latent geometry.
Paper Structure (58 sections, 6 theorems, 32 equations, 2 figures, 3 tables)

This paper contains 58 sections, 6 theorems, 32 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation and Conventions
  4. Continuous-Time State-Space Models
  5. Discretization via Induced Dynamics
  6. Fractals and Self-Similarity
  7. Halting and Adaptive Computation
  8. Fractal of Stationary Transformations
  9. Scale-Consistent Refinement
  10. Stationary Transformations and Discretization
  11. Fractal Inductive Bias via Self-Similarity
  12. Scaled State Transition.
  13. Discrete Realization.
  14. Output Scaling.
  15. Variable-Complexity Inference
  16. ...and 43 more sections

Key Result

Lemma 2.1

Let $h(\tau)$ satisfy the induced dynamics on $[t,t+1)$ with initial condition $h(t)$. Then

Figures (2)

  • Figure 1: Convergence behavior of $\lambda$ across different $H$.
  • Figure 2: Convergence trajectory of $s$ varying $\mathcal{L}_\text{rank}$ composition.

Theorems & Definitions (14)

  • Lemma 2.1: Exact discrete update of the induced dynamics
  • proof
  • Definition 3.1: Scale-consistent refinement
  • Definition 3.2: Stationarity and contractivity
  • Lemma 3.3: Scaling equivariance of induced dynamics
  • Proposition 3.4: Fractal inductive bias of stationary transformations
  • Definition 2.1: FROST State Update
  • Theorem 2.2: Contraction Mapping Principle
  • proof
  • Theorem 2.3: Exponential Error Decay
  • ...and 4 more