A Data-Driven Prism: Multi-View Source Separation with Diffusion Model Priors

TL;DR

The paper tackles multi-view source separation (MVSS) where the underlying sources are unknown and observations are noisy and incomplete. It introduces DDPRISM, a data-driven framework that learns independent diffusion-prior models for each source $p(\mathbf{x}^\beta)$ and uses an EM loop to sample the joint posterior given observations and known mixing matrices. Key contributions include a generalist MVSS approach that does not require explicit source priors, a joint diffusion-posterior sampling scheme via MMPS approximations, and state-of-the-art performance on contrastive MVSS as well as real-world galaxy imaging tasks. The approach enables principled uncertainty quantification and posterior sampling in challenging scientific settings, though it incurs substantial computational costs and relies on linear mixing and Gaussian noise assumptions.

Abstract

A common challenge in the natural sciences is to disentangle distinct, unknown sources from observations. Examples of this source separation task include deblending galaxies in a crowded field, distinguishing the activity of individual neurons from overlapping signals, and separating seismic events from an ambient background. Traditional analyses often rely on simplified source models that fail to accurately reproduce the data. Recent advances have shown that diffusion models can directly learn complex prior distributions from noisy, incomplete data. In this work, we show that diffusion models can solve the source separation problem without explicit assumptions about the source. Our method relies only on multiple views, or the property that different sets of observations contain different linear transformations of the unknown sources. We show that our method succeeds even when no source is individually observed and the observations are noisy, incomplete, and vary in resolution. The learned diffusion models enable us to sample from the source priors, evaluate the probability of candidate sources, and draw from the joint posterior of the source distribution given an observation. We demonstrate the effectiveness of our method on a range of synthetic problems as well as real-world galaxy observations.

Figures (5)

• Figure 1: Comparison of posterior samples for our joint sampling method and the Gibbs sampling method heurtel_depeiges2024listening on the 1D manifold problem. Both methods are equivalent for the first source. The plots show the evolution of the marginals for the first and second dimension of the specified source distribution. The last EM lap for sources $\beta=1,2,3$ are $16,32,64$ respectively.
• Figure 2: Comparison of the mean Sinkhorn divergence for different $f_\text{mix}$ on the mixed 1D manifold problem. Also shown are the Gibbs sampling method with eight times as many computations per EM lap and our method when $f_\text{mix}=1.0$ and $A_{i_\alpha}^{\alpha\beta}$ depends on $\beta$. Even for large mixing fractions, our method can accurately learn the two distinct underlying source distributions.
• Figure 3: Comparison of posterior samples for two example observations in Grassy MNIST experiment. The observations are on the far left and right, the true input sources are in the middle, and a draw from DDPRISM [ours], CLVM-VAE clvm, and PCPCA pcpca for both the full-resolution and downsampled case is shown in between. PCPCA cannot sample the grass posterior, and CLVM-Linear is omitted for brevity. Our joint diffusion model returns the best reconstruction of both sources, with near-perfect posterior samples in the full-resolution case.
• Figure 4: Observations of galaxy images along with posterior samples for the random and galaxy source using our method. Images are split into high-contrast (upper region) and low-contrast (lower region) colormaps to highlight the range of features. The galaxy source captures the central light while the small-scale fluctuations and the uncorrelated light is separated into the random source.
• Figure 5: Random samples of galaxy posteriors across EM iterations in the galaxy–image experiment. Early iterations fail to isolate galaxy light and show strong small-scale fluctuations. By iteration 4, small-scale fluctuations are separated but residual uncorrelated light remains. By iteration 16, the uncorrelated light is assigned to the random posterior component, leaving nearly noiseless, isolated galaxy posteriors. As in Figure \ref{['fig:gal']}, images are split into high-contrast and low-contrast colormaps to highlight the range of features.