Continuous Domain Generalization

TL;DR

This work tackles the challenge of generalizing to unseen domains that shift continuously across multiple latent factors, beyond single-axis or discrete domain assumptions. It posits that the optimal domain-specific parameters lie on a low-dimensional parameter manifold and introduces NeuralLio, a Lie-group–inspired transport operator that enforces geometry-aware, algebraically consistent parameter transitions across domain descriptors. To cope with real-world descriptor imperfections, the framework adds descriptor gating and a local-chart atlas to ensure robust, smooth generalization. Across synthetic and real-world datasets, NeuralLio achieves stronger generalization and robustness than baselines, with empirical evidence supporting the manifold view and the efficacy of the structural constraints. The approach has broad implications for scalable, context-aware domain generalization in multimodal and spatiotemporal settings.

Abstract

Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e.g., time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

Figures (9)

• Figure 1: Illustration of continuous domain generalization. Real-world distributions are shaped by continuously varying factors. The observed domains provide sparse supervision over the joint variation-model space. Continuous domain generalization studies how predictive functions evolve over this space, enabling the model to generalize to unseen domains across the underlying continuous fields.
• Figure 2: Visualization of generalization behavior of baseline models on the 2-Moons dataset. Left: All training domains (black dots) and selected test domains (blue crosses) in the variation space. Right: Decision boundaries of baseline methods (rows) evaluated at the four test domains (columns).
• Figure 3: Visualization of the learned parameter manifold and the corresponding generalization behavior. Left: PCA projection of predicted parameters $\theta(z)$ over the entire descriptor space. Right: Visualization of decision boundaries and data samples along selected direction.
• Figure 4: Robustness to imperfect descriptors. Top: Visualization of noisy, redundant, and incomplete descriptor constructions. Bottom: Error rates under increasing imperfection on the 2-Moons dataset.
• Figure 5: Train and test domain descriptors for the 2-Moons dataset. The left plot shows the 50 training domains, while the right plot shows the 150 randomly sampled test domains and additional test domains uniformly distributed over a fixed mesh grid in the descriptor space.

Theorems & Definitions (4)

• Theorem 1: Parameter Manifold
• proof : Proof
• Definition 1: Geometric Structure
• Definition 2: Algebraic Structure