Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Renormalization Group Guided Tensor Network Structure Search

Maolin Wang, Bowen Yu, Sheng Zhang, Linjie Mi, Wanyu Wang, Yiqi Wang, Pengyue Jia, Xuetao Wei, Zenglin Xu, Ruocheng Guo, Xiangyu Zhao

TL;DR

TN-SS methods struggle with tractable search over variable tensor-network topologies, single-scale optimization, and separation of structure from parameter learning. RGTN introduces Renormalization Group-guided multi-scale flows to continuously evolve network topology across resolutions, using learnable edge gates and physics-inspired metrics (node tension, edge information flow) to jointly optimize structure and parameters. Theoretical results establish convergence guarantees, structure-recovery under mild noise, and exponential speedups over sampling-based approaches, along with high-probability escape from local minima. Empirically, RGTN achieves state-of-the-art compression and orders-of-magnitude faster runtimes on light-field data, high-order tensors, and video completion, demonstrating robust performance across diverse tensor modalities.

Abstract

Tensor network structure search (TN-SS) aims to automatically discover optimal network topologies and rank configurations for efficient tensor decomposition in high-dimensional data representation. Despite recent advances, existing TN-SS methods face significant limitations in computational tractability, structure adaptivity, and optimization robustness across diverse tensor characteristics. They struggle with three key challenges: single-scale optimization missing multi-scale structures, discrete search spaces hindering smooth structure evolution, and separated structure-parameter optimization causing computational inefficiency. We propose RGTN (Renormalization Group guided Tensor Network search), a physics-inspired framework transforming TN-SS via multi-scale renormalization group flows. Unlike fixed-scale discrete search methods, RGTN uses dynamic scale-transformation for continuous structure evolution across resolutions. Its core innovation includes learnable edge gates for optimization-stage topology modification and intelligent proposals based on physical quantities like node tension measuring local stress and edge information flow quantifying connectivity importance. Starting from low-complexity coarse scales and refining to finer ones, RGTN finds compact structures while escaping local minima via scale-induced perturbations. Extensive experiments on light field data, high-order synthetic tensors, and video completion tasks show RGTN achieves state-of-the-art compression ratios and runs 4-600$\times$ faster than existing methods, validating the effectiveness of our physics-inspired approach.

Renormalization Group Guided Tensor Network Structure Search

TL;DR

TN-SS methods struggle with tractable search over variable tensor-network topologies, single-scale optimization, and separation of structure from parameter learning. RGTN introduces Renormalization Group-guided multi-scale flows to continuously evolve network topology across resolutions, using learnable edge gates and physics-inspired metrics (node tension, edge information flow) to jointly optimize structure and parameters. Theoretical results establish convergence guarantees, structure-recovery under mild noise, and exponential speedups over sampling-based approaches, along with high-probability escape from local minima. Empirically, RGTN achieves state-of-the-art compression and orders-of-magnitude faster runtimes on light-field data, high-order tensors, and video completion, demonstrating robust performance across diverse tensor modalities.

Abstract

Tensor network structure search (TN-SS) aims to automatically discover optimal network topologies and rank configurations for efficient tensor decomposition in high-dimensional data representation. Despite recent advances, existing TN-SS methods face significant limitations in computational tractability, structure adaptivity, and optimization robustness across diverse tensor characteristics. They struggle with three key challenges: single-scale optimization missing multi-scale structures, discrete search spaces hindering smooth structure evolution, and separated structure-parameter optimization causing computational inefficiency. We propose RGTN (Renormalization Group guided Tensor Network search), a physics-inspired framework transforming TN-SS via multi-scale renormalization group flows. Unlike fixed-scale discrete search methods, RGTN uses dynamic scale-transformation for continuous structure evolution across resolutions. Its core innovation includes learnable edge gates for optimization-stage topology modification and intelligent proposals based on physical quantities like node tension measuring local stress and edge information flow quantifying connectivity importance. Starting from low-complexity coarse scales and refining to finer ones, RGTN finds compact structures while escaping local minima via scale-induced perturbations. Extensive experiments on light field data, high-order synthetic tensors, and video completion tasks show RGTN achieves state-of-the-art compression ratios and runs 4-600 faster than existing methods, validating the effectiveness of our physics-inspired approach.
Paper Structure (58 sections, 8 theorems, 62 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 58 sections, 8 theorems, 62 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Theoretical Foundation
  4. Scale-Dependent Effective Action
  5. Enhanced Tensor Network Architecture
  6. Adaptive Diagonal Factors
  7. Edge Gating Mechanism
  8. Multi-Scale Loss Function
  9. Intelligent Structure Search via RG Flow
  10. Smart Proposal Generation
  11. Adaptive Optimization Strategy
  12. Structure Discovery through Sparsity
  13. Multi-Scale Progressive Refinement
  14. Theoretical Analysis
  15. Preliminaries and Assumptions
  16. ...and 43 more sections

Key Result

Theorem 1

Under Assumptions assum:lipschitz-assum:bounded_params, the RGTN algorithm generates a sequence of networks $\{\mathcal{M}^{(t)}\}_{t=0}^{\infty}$ that converges to a critical point of the multi-scale objective function. Specifically, for any $\epsilon > 0$, there exists $T(\epsilon)$ such that for Moreover, the convergence rate satisfies: where $\eta_t$ are the learning rates and $\mathcal{L}^*

Figures (6)

  • Figure 1: The RGTN framework transforms network topology through physics-inspired multi-scale operations instead of traditional sampling-evaluation methods. It processes tensors through three RG-based phases: coarse-graining (downsampling and tension calculation), expansion (splitting high-tension nodes), and compression (merging low-flow edges), unifying structure search and parameter optimization for efficient discovery of optimal tensor network structures.
  • Figure 2: Reconstructed images and residual images obtained by different methods (including ours) on the 25th frame of News. Here the residual image is the average absolute difference between the reconstructed image and the ground truth over R, G, and B channels.
  • Figure 3: Hyperparameter analysis for the video completion task, showing the impact of varying bond dimension on performance. The plots show: (Top-left) Mean Peak Signal-to-Noise Ratio (MPSNR) vs. Bond Dimension; (Top-right) Peak Signal-to-Noise Ratio (PSNR) vs. Bond Dimension; (Bottom-left) The effect of bond dimension on total runtime; (Bottom-right) A trade-off analysis between MPSNR and runtime, with bond dimension encoded by color and labels. The results indicate that a bond dimension of 12.0 achieves the highest MPSNR (32.04 dB), suggesting an optimal configuration for this task. Increasing the bond dimension further leads to significantly longer runtimes without a corresponding improvement in reconstruction quality.
  • Figure 4: Convergence behavior of the RGTN algorithm on the Bunny and Silent test video data. The vertical axis represents the relative change between consecutive iterations, defined as $\|\mathcal{X}^{(k)} - \mathcal{X}^{(k-1)}\|_F / \|\mathcal{X}^{(k-1)}\|_F$, where $\mathcal{X}^{(k)}$ is the reconstructed tensor at the k-th iteration. The plots demonstrate that our RGTN algorithm converges rapidly within the initial iterations and maintains excellent stability throughout the optimization process.
  • Figure 5: Comparison of runtime efficiency between our RGTN multi-scale initialization strategy and a standard random initialization on the 'Bunny' light field data. The chart shows the time required for both initialization schemes to reach three different reconstruction error (RE) targets. The results clearly demonstrate that our physics-inspired, coarse-to-fine initialization provides an effective 'warm start', dramatically reducing the optimization time needed to achieve the desired accuracy compared to starting from a random state.
  • ...and 1 more figures

Theorems & Definitions (24)

  • Theorem 1: Global Convergence of Multi-Scale Optimization
  • proof
  • Lemma 1: Diagonal Factor Sparsity Pattern
  • proof
  • Theorem 2: Structure Recovery Guarantee
  • proof
  • Theorem 3: Computational Complexity Comparison
  • proof
  • Lemma 2: Loss Landscape Smoothing
  • proof
  • ...and 14 more