Zero-Flow Encoders

TL;DR

Zero-Flow Encoders leverage a theoretical zero-flow phenomenon in rectified flow to certify conditional independence and learn sufficient encodings without parametric density models. By translating the zero-flow criterion into a simulation-free loss, the method enables amortized Markov blanket learning and robust self-supervised representations with principled guarantees. Empirical results on graphical models, time-series (e.g., S&P500), and vision benchmarks demonstrate accurate structure recovery and resilience to shortcuts. The work provides a novel, nonparametric criterion for sufficiency and conditional independence with practical learning algorithms and broad applicability.

Abstract

Flow-based methods have achieved significant success in various generative modeling tasks, capturing nuanced details within complex data distributions. However, few existing works have exploited this unique capability to resolve fine-grained structural details beyond generation tasks. This paper presents a flow-inspired framework for representation learning. First, we demonstrate that a rectified flow trained using independent coupling is zero everywhere at $t=0.5$ if and only if the source and target distributions are identical. We term this property the \emph{zero-flow criterion}. Second, we show that this criterion can certify conditional independence, thereby extracting \emph{sufficient information} from the data. Third, we translate this criterion into a tractable, simulation-free loss function that enables learning amortized Markov blankets in graphical models and latent representations in self-supervised learning tasks. Experiments on both simulated and real-world datasets demonstrate the effectiveness of our approach. The code reproducing our experiments can be found at: https://github.com/probabilityFLOW/zfe.

Figures (11)

• Figure 1: "Zero-flow" phenomenon. Rectified flow trained on identical distributions (a Gaussian mixture) with independent coupling. At $t=0.5$, the flow becomes stationary almost everywhere. An explanation of this phenomenon can be found in Section \ref{['sec.zeroflow']}, and additional visualizations can be found in Figure \ref{['fig:app_random']}.
• Figure 2: Left: $X = 0.5Y + \mathcal{N}(0,1)$. Right: $f(Y) = \sigma(-2Y)$ is a sufficient statistic for predicting $X$, thus $\mathbf{v}_{t=0.5} = 0$ almost everywhere as predicted by Theorem \ref{['eq.thm.zero.cond.flow']}. Center: $f(Y) = \sin(2Y)$ is not a sufficient statistic for predicting $X$, thus $\mathbf{v}_{t=0.5} \neq 0$, as predicted by Theorem \ref{['eq.thm.zero.cond.flow']}. The vector fields are trained using a two-layer ReLU network with an empirical version of \ref{['eq.cond.rf']}.
• Figure 3: Left: Saturated SimCLR loss. On the watermarked CIFAR10 dataset, the loss quickly collapses to small values due to shortcuts. Right: The loss of the proposed method. The loss remains steady before and after the watermark is introduced.
• Figure 4: The comparison of ROC curves in terms of graphical model structure recovery on truncated-Gaussian (left) and non-paranormal (right) graphical models.
• Figure 5: The sudden change of Markov blanket patterns, before and after March 2020. Before, Markov blankets consisted of past trading days. Immediately after, Markov blankets predominantly consisted of future trading days.

Theorems & Definitions (12)

• Theorem 3.1: Zero-flow Condition
• Theorem 3.2: Conditional Transport
• Theorem 3.3: Conditional Zero-flow Condition
• proof
• Lemma 2.1
• proof
• Theorem 2.2: Restatement \ref{['eq.thm.cond.transport']}
• proof
• Theorem 2.3: Conditional Zero-flow Condition Restatement
• proof