Phase-space entropy at acquisition reflects downstream learnability

TL;DR

The paper proposes a modality-agnostic, acquisition-time scalar ΔS_B based on instrument-resolved phase-space entropy to quantify how data collection preserves or disrupts structures leveraged by downstream learners. By constructing a Husimi density ρ_I and computing band-entropy changes within a Nyquist band, the authors show that coherent (periodic) sampling increases entropy through spectral folding, while random sampling preserves it in expectation. Across vision, accelerated MRI, and massive MIMO—with both simulations and over-the-air experiments—|ΔS_B| ranks sampling geometries and predicts downstream reconstruction/recognition difficulty without training; notably, minimizing |ΔS_B| enables zero-training design in MRI. The work provides a unified, physics-informed framework for pre-training acquisition design and a shared notion of information preservation across sensing modalities, with practical workflow recommendations and clear pathways for extension.

Abstract

Modern learning systems work with data that vary widely across domains, but they all ultimately depend on how much structure is already present in the measurements before any model is trained. This raises a basic question: is there a general, modality-agnostic way to quantify how acquisition itself preserves or destroys the information that downstream learners could use? Here we propose an acquisition-level scalar $ΔS_{\mathcal B}$ based on instrument-resolved phase space. Unlike pixelwise distortion or purely spectral errors that often saturate under aggressive undersampling, $ΔS_{\mathcal B}$ directly quantifies how acquisition mixes or removes joint space--frequency structure at the instrument scale. We show theoretically that \(ΔS_{\mathcal B}\) correctly identifies the phase-space coherence of periodic sampling as the physical source of aliasing, recovering classical sampling-theorem consequences. Empirically, across masked image classification, accelerated MRI, and massive MIMO (including over-the-air measurements), $|ΔS_{\mathcal B}|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty \emph{without training}. In particular, minimizing $|ΔS_{\mathcal B}|$ enables zero-training selection of variable-density MRI mask parameters that matches designs tuned by conventional pre-reconstruction criteria. These results suggest that phase-space entropy at acquisition reflects downstream learnability, enabling pre-training selection of candidate sampling policies and as a shared notion of information preservation across modalities.

Figures (3)

• Figure 1: Phase-space entropy uniquely discriminates the advantage of random over periodic sampling.(a) Visual illustration of the sampling geometries, showing the original fully sampled signal (left), a periodic sampling mask (center), and a random sampling mask (right). (b) Violin plots quantifying the comparative advantage provided by Random sampling over Periodic sampling across the test set for three metrics: $\Delta S_{\mathcal{B}}$ (red), PSNR (blue), and Spectral $L_2$ (orange). The y-axis represents the normalized advantage (e.g., $\Delta S_{\mathcal{B}}^{\text{periodic}} - \Delta S_{\mathcal{B}}^{\text{random}}$); higher positive values indicate a stronger preference for the Random geometry. Inner dashed lines mark the quartiles. While PSNR and Spectral $L_2$ advantages concentrate near zero and show weak sensitivity to sparsity changes, the $\Delta S_{\mathcal{B}}$ advantage is strictly positive and becomes increasingly pronounced as sparsity rises ($k=2 \to 8$). This confirms that phase-space entropy effectively captures the geometry-induced learnability gap that conventional pre-training proxies tend to compress.
• Figure 2: Band-entropy change under $k$-space subsampling predicts reconstruction quality in accelerated MRI.(a) Schematic of accelerated acquisition: binary masks along the phase-encoding ($k_y$) direction remove Fourier lines outside a central auto-calibration region. Zero-filled inverse FFT produces aliased magnitude images that serve as inputs to a U-Net. (b) Reconstruction PSNR/SSIM and average $|\Delta S_{\mathcal{B}}|$ for six mask families across acceleration factors. Canonical masks (Periodic, Random, Poisson) recover the familiar ranking, with Poisson achieving both the best reconstruction quality and the smallest $|\Delta S_{\mathcal{B}}|$. Three variable-density Cartesian designs are selected in a pre-training way by optimizing, respectively, $|\Delta S_{\mathcal{B}}|$, $k$-space $L_2$ error, or pre-reconstruction PSNR; all three attain strong PSNR/SSIM, and the $|\Delta S_{\mathcal{B}}|$-optimized mask consistently yields the smallest entropy perturbation. (c) Violin plots of test-set PSNR, SSIM, and $|\Delta S_{\mathcal{B}}|$ for all six masks. Reconstruction metrics are tightly clustered and show limited separation between acquisition geometries, whereas $|\Delta S_{\mathcal{B}}|$ exhibits pronounced differences in both mean and distribution shape, indicating that phase-space entropy reveals latent structure in acquisition-induced information loss beyond what standard image-domain metrics capture.
• Figure 3: Entropy auditing of MIMO channels and over-the-air validation.(a) NMSE of CSI reconstruction versus mean band-entropy change $\Delta S_{\mathcal{B}}$ for Periodic and random antenna deactivation at different pilot budgets in simulation. Each point corresponds to a fixed deactivation interval $d$ and geometry; larger $\Delta S_{\mathcal{B}}$ is associated with higher NMSE. (b) Violin plots of $\Delta S_{\mathcal{B}}$ across simulated channels for each deactivation scheme and budget. Periodic deactivation consistently yields larger entropy perturbations than random deactivation at matched budgets. (c) Schematic of the OTA testbed with a $1\times 8$ ULA at the transmitter and a $1\times 8$ ULA at the receiver, forming an $8\times 8$ MIMO link. (d) OTA results: random deactivation achieves lower NMSE and smaller $|\Delta S_{\mathcal{B}}|$ than periodic deactivation under the same pilot budget.