Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Breaking Scale Anchoring: Frequency Representation Learning for Accurate High-Resolution Inference from Low-Resolution Training

Wenshuo Wang, Fan Zhang

TL;DR

This work identifies Scale Anchoring as an information-theoretic limitation that prevents true cross-resolution generalization in zero-shot spatiotemporal forecasting. It introduces Frequency Representation Learning (FRL), an architecture-agnostic framework that normalizes frequencies across resolutions and enforces spectral consistency to mitigate high-frequency extrapolation gaps. Theoretical analysis links Nyquist-bound learning to frequency blindness and validates this mechanism with experiments across 3D fluid simulations and ERA5-based weather forecasting. Empirically, FRL yields substantial high-resolution accuracy gains with modest training-time and memory overhead, demonstrating its potential to enable reliable, multi-resolution inference in STF while clarifying the boundaries of its applicability.

Abstract

Zero-Shot Super-Resolution Spatiotemporal Forecasting requires a deep learning model to be trained on low-resolution data and deployed for inference on high-resolution. Existing studies consider maintaining similar error across different resolutions as indicative of successful multi-resolution generalization. However, deep learning models serving as alternatives to numerical solvers should reduce error as resolution increases. The fundamental limitation is, the upper bound of physical law frequencies that low-resolution data can represent is constrained by its Nyquist frequency, making it difficult for models to process signals containing unseen frequency components during high-resolution inference. This results in errors being anchored at low resolution, incorrectly interpreted as successful generalization. We define this fundamental phenomenon as a new problem distinct from existing issues: Scale Anchoring. Therefore, we propose architecture-agnostic Frequency Representation Learning. It alleviates Scale Anchoring through resolution-aligned frequency representations and spectral consistency training: on grids with higher Nyquist frequencies, the frequency response in high-frequency bands of FRL-enhanced variants is more stable. This allows errors to decrease with resolution and significantly outperform baselines within our task and resolution range, while incurring only modest computational overhead.

Breaking Scale Anchoring: Frequency Representation Learning for Accurate High-Resolution Inference from Low-Resolution Training

TL;DR

This work identifies Scale Anchoring as an information-theoretic limitation that prevents true cross-resolution generalization in zero-shot spatiotemporal forecasting. It introduces Frequency Representation Learning (FRL), an architecture-agnostic framework that normalizes frequencies across resolutions and enforces spectral consistency to mitigate high-frequency extrapolation gaps. Theoretical analysis links Nyquist-bound learning to frequency blindness and validates this mechanism with experiments across 3D fluid simulations and ERA5-based weather forecasting. Empirically, FRL yields substantial high-resolution accuracy gains with modest training-time and memory overhead, demonstrating its potential to enable reliable, multi-resolution inference in STF while clarifying the boundaries of its applicability.

Abstract

Zero-Shot Super-Resolution Spatiotemporal Forecasting requires a deep learning model to be trained on low-resolution data and deployed for inference on high-resolution. Existing studies consider maintaining similar error across different resolutions as indicative of successful multi-resolution generalization. However, deep learning models serving as alternatives to numerical solvers should reduce error as resolution increases. The fundamental limitation is, the upper bound of physical law frequencies that low-resolution data can represent is constrained by its Nyquist frequency, making it difficult for models to process signals containing unseen frequency components during high-resolution inference. This results in errors being anchored at low resolution, incorrectly interpreted as successful generalization. We define this fundamental phenomenon as a new problem distinct from existing issues: Scale Anchoring. Therefore, we propose architecture-agnostic Frequency Representation Learning. It alleviates Scale Anchoring through resolution-aligned frequency representations and spectral consistency training: on grids with higher Nyquist frequencies, the frequency response in high-frequency bands of FRL-enhanced variants is more stable. This allows errors to decrease with resolution and significantly outperform baselines within our task and resolution range, while incurring only modest computational overhead.

Paper Structure

This paper contains 47 sections, 5 theorems, 39 equations, 6 figures, 34 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Zero-Shot Super-Resolution Spatiotemporal Forecasting
  4. Scale Anchoring V.S. Related Phenomena
  5. Existence of Scale Anchoring
  6. Mechanism of Scale Anchoring
  7. Theoretical Analysis
  8. Experiment Validation
  9. Solution for Scale Anchoring
  10. Empirical Evaluation
  11. Experimental setup
  12. Datasets
  13. Baseline Selection and Implementation Details
  14. Metrics
  15. Zero-Shot Super-Resolution Spatiotemporal Forecasting
  16. ...and 32 more sections

Key Result

Theorem 1

A neural network trained at resolution $\rho_0$ cannot correctly process frequency components above the Nyquist frequency $\rho_0/2$. The learned operator's frequency response satisfies:

Figures (6)

  • Figure 1: High and low resolution RMSE with their ratio across different methods in 3D ZS-SR fluid simulation.
  • Figure 2: Frequency response analysis on different architectures, models trained at resolution $64^2$.
  • Figure 3: Loss of different frequency components during propagation.
  • Figure 4:
  • Figure 5:
  • ...and 1 more figures

Theorems & Definitions (10)

  • Theorem 1: Frequency Blindness
  • Theorem 2: High-Frequency Error Dominance
  • Definition 1: Domain and Sampling
  • Definition 2: Interpolation Operators
  • Theorem 3: Frequency Blindness
  • proof
  • Theorem 4: High-Frequency Error Dominance
  • proof
  • Corollary 5: $\text{RMSE}_{\text{Ratio}}$ Behavior
  • proof