Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networks

Abstract

Efficient deep learning traditionally relies on static heuristics like weight magnitude or activation awareness (e.g., Wanda, RIA). While successful in unstructured settings, we observe a critical limitation when applying these metrics to the structural pruning of deep vision networks. These contemporary metrics suffer from a magnitude bias, failing to preserve critical functional pathways. To overcome this, we propose a decoupled kinetic paradigm inspired by Alternating Gradient Flow (AGF), utilizing an absolute feature-space Taylor expansion to accurately capture the network's structural "kinetic utility". First, we uncover a topological phase transition at extreme sparsity, where AGF successfully preserves baseline functionality and exhibits topological implicit regularization, avoiding the collapse seen in models trained from scratch. Second, transitioning to architectures without strict structural priors, we reveal a phenomenon of Sparsity Bottleneck in Vision Transformers (ViTs). Through a gradient-magnitude decoupling analysis, we discover that dynamic signals suffer from signal compression in converged models, rendering them suboptimal for real-time routing. Finally, driven by these empirical constraints, we design a hybrid routing framework that decouples AGF-guided offline structural search from online execution via zero-cost physical priors. We validate our paradigm on large-scale benchmarks: under a 75% compression stress test on ImageNet-1K, AGF effectively avoids the structural collapse where traditional metrics aggressively fall below random sampling. Furthermore, when systematically deployed for dynamic inference on ImageNet-100, our hybrid approach achieves Pareto-optimal efficiency. It reduces the usage of the heavy expert by approximately 50% (achieving an estimated overall cost of 0.92$\times$) without sacrificing the full-model accuracy.

Figures (6)

• Figure 1: Overview of the AGF-Guided Efficiency Framework. The pipeline integrates (A) Gradient-based Utility Calibration, (B) Iterative Structural Pruning, and (C) Confidence-based Dynamic Routing. AGF identifies the topological skeleton, while the routing policy handles runtime complexity.
• Figure 2: Analysis of Metric Stability and Orthogonality (WideResNet on CIFAR-100 at $k=32$). (a) Selection Stability: AGF demonstrates superior batch-to-batch structural consistency compared to the Taylor baseline. (b) Metric Orthogonality ($J \approx 0$): A normalized scatter plot reveals a fundamental divergence between static and dynamic metrics. Traditional magnitude metrics ($\ell_1$) rigidly select high-capacity channels (blue crosses), whereas AGF identifies an entirely distinct, orthogonal subset of dynamic routing hubs with high kinetic potential (red dots).
• Figure 3: Difficulty Distribution of Routed Samples. We measure sample difficulty using the prediction entropy of the full-capacity baseline. The lightweight router successfully learns to disentangle the input space without human priors: "easy" samples (low entropy) are dominantly routed to the Pruned Expert (Green), while "hard" ambiguous samples (high entropy) are forwarded to the Full Expert (Red). This adaptive decoupling is the core mechanism enabling our system's high efficiency.
• Figure 4: Accuracy-Efficiency Trade-off on ImageNet-100. Our adaptive method (Red) establishes a superior Pareto frontier compared to static and random baselines, enabling dynamic trade-offs between performance and cost.
• Figure 5: Qualitative Visualization of Routing Decisions. Top row: "Easy" samples (e.g., clear objects) routed to the Pruned Expert. Bottom row: "Hard" samples (e.g., clutter) routed to the Full Expert. The router effectively identifies samples requiring higher capacity for correct classification.