Learning Domain-Invariant Representations for Cross-Domain Image Registration via Scene-Appearance Disentanglement

TL;DR

SAR-Net addresses image registration under domain shift by factorizing observed images into a domain-invariant scene representation and a domain-specific appearance code. Registration then proceeds via cross-domain re-rendering in a shared latent scene space, supported by theoretical conditions that guarantee geometric consistency when the scene is preserved. The method integrates a Scene Encoder, an Appearance Encoder, and a Forward Model to jointly reconstruct and align images across domains, with a scene-consistency loss providing a sufficient condition for cross-domain alignment. Empirically on bidirectional OR-PAM data, SAR-Net delivers a 3.1x improvement in SSIM over the strongest baseline, runs in real time at ~77 fps, and shows robust temporal consistency and vascular-structure preservation, with ablations confirming the necessity of each component.

Abstract

Image registration under domain shift remains a fundamental challenge in computer vision and medical imaging: when source and target images exhibit systematic intensity differences, the brightness constancy assumption underlying conventional registration methods is violated, rendering correspondence estimation ill-posed. We propose SAR-Net, a unified framework that addresses this challenge through principled scene-appearance disentanglement. Our key insight is that observed images can be decomposed into domain-invariant scene representations and domain-specific appearance codes, enabling registration via re-rendering rather than direct intensity matching. We establish theoretical conditions under which this decomposition enables consistent cross-domain alignment (Proposition 1) and prove that our scene consistency loss provides a sufficient condition for geometric correspondence in the shared latent space (Proposition 2). Empirically, we validate SAR-Net on bidirectional scanning microscopy, where coupled domain shift and geometric distortion create a challenging real-world testbed. Our method achieves 0.885 SSIM and 0.979 NCC, representing 3.1x improvement over the strongest baseline, while maintaining real-time performance (77 fps). Ablation studies confirm that both scene consistency and domain alignment losses are necessary: removing either degrades performance by 90% SSIM or causes 223x increase in latent alignment error, respectively. Code and data are available at https://github.com/D-ST-Sword/SAR-NET.

Figures (6)

• Figure 1: Overview of the proposed SAR-Net framework. (I) Network architecture: Scene Encoder $E_S$ extracts domain-invariant anatomical structure using instance normalization; Appearance Encoder $E_A$ captures domain-specific acquisition parameters via global average pooling; Forward Model $G$ synthesizes images through feature modulation. The scene consistency loss $\mathcal{L}_{\text{scene}}$ enforces geometric alignment between $S_{\text{odd}}$ and $S_{\text{even}}$. (II) Comparison of registration pipelines: Conventional deep learning registration methods directly align images under brightness constancy assumptions, whereas our approach maps all frames to a shared scene space where $S_1 \approx S_2 \approx \cdots \approx S_n$, enabling implicit inter-frame alignment without explicit frame-to-frame registration. (III) Registration results: Before/after comparison with odd-even overlay visualization (odd in magenta, even in green, alignment as white) demonstrating effective column alignment.
• Figure 2: Qualitative comparison of registration results. Visual comparison on a representative frame across all ten methods. For each method: interleaved images with HOT colormap, odd-even overlay (odd in magenta, even in green, alignment as white), and grayscale visualization. Cross-frame overlay (rightmost) shows inter-frame alignment quality. Our method achieves continuous vascular structures with minimal color separation in overlays.
• Figure 3: Quantitative comparison across three mice. SSIM (top) and NCC (bottom) box plots showing metric distributions across multiple subjects. SAR-Net consistently outperforms all baseline methods.
• Figure 4: ROI selection for vascular continuity analysis. Three representative regions (A, B, C) are selected from the interleaved image to evaluate registration quality across different vascular structures.
• Figure 5: Vessel Continuity Index (VCI) comparison across three ROIs. Higher VCI indicates better column alignment and vascular continuity. SAR-Net substantially outperforms all baseline methods.

Theorems & Definitions (3)

• Proposition 1: Cross-Domain Alignment via Re-Rendering
• proof
• Proposition 2: Sufficiency of Scene Consistency