CURVE: Learning Causality-Inspired Invariant Representations for Robust Scene Understanding via Uncertainty-Guided Regularization

TL;DR

CURVE tackles the challenge of robust scene understanding under distribution shifts by explicitly modeling a causal split between invariant factors $z_c$ and environment-dependent confounds $z_s$, and by leveraging variational uncertainty to guide a soft, prototype-driven backdoor intervention. The method combines a probabilistic scene-graph generator with differentiable structure learning to prune spurious edges, yielding a sparse, domain-stable topology that emphasizes invariant causal dynamics. Its core contributions are a prototype-based environment approximation for backdoor adjustment, an uncertainty-guided sparsification mechanism, and an integrated objective that calibrates uncertainty while promoting diversity across prototypes. Empirically, CURVE improves zero-shot OOD generalization and sim-to-real transfer in autonomous-driving risk tasks, while providing calibrated uncertainty estimates to support risk assessment under distribution shifts, highlighting its potential for safety-critical scene understanding.

Abstract

Scene graphs provide structured abstractions for scene understanding, yet they often overfit to spurious correlations, severely hindering out-of-distribution generalization. To address this limitation, we propose CURVE, a causality-inspired framework that integrates variational uncertainty modeling with uncertainty-guided structural regularization to suppress high-variance, environment-specific relations. Specifically, we apply prototype-conditioned debiasing to disentangle invariant interaction dynamics from environment-dependent variations, promoting a sparse and domain-stable topology. Empirically, we evaluate CURVE in zero-shot transfer and low-data sim-to-real adaptation, verifying its ability to learn domain-stable sparse topologies and provide reliable uncertainty estimates to support risk prediction under distribution shifts.

Figures (5)

• Figure 1: CURVE mitigates OOD failures by counteracting environment-driven dense edges. The top panels illustrate OOD failures in latent/dense graph approaches, as dense connectivity induces environment-conditioned edges, encouraging shortcut reliance on background co-occurrences. The bottom panel demonstrates our method using soft causal backdoor adjustment and uncertainty-driven sparsification to separate factors and produce a sparse, robust graph.
• Figure 2: Overview of CURVE framework. Our approach disentangles invariant causal factors $z_c$ from environmental bias factors $z_s$. The core mechanism employs data-dependent uncertainty $\sigma_{ij}$ to reweight a causal intervention, leveraging a learnable prototype dictionary $\mathcal{C}$ to estimate and compensate for environment-conditioned bias. Finally, differentiable pruning eliminates spurious correlations to yield a sparse causality-consistent graph for robust reasoning and calibrated prediction.
• Figure 3: Generalization Analysis (RQ1). (a-d) Structural Sparsity. While the baseline RS2G retains dense connectivity by aggregating diverse redundant correlations between nodes, CURVE utilizes causal intervention to prune spurious edges driven by the environment. (e) Data Efficiency. Adaptation performance (AUC) from simulation to the real world. The results demonstrate the transferability of CURVE, which maintains high accuracy even with minimal annotations.
• Figure 4: Visualization of the learned features. The t-SNE projection reveals a risk evolution manifold that reflects the temporal dynamics of accidents. The trajectory evolves from concentrated stable safe states, through a critical transition phase where risk accumulates, to inevitable collision states, demonstrating the model's capacity to encode physical progression.
• Figure 5: Reliability and Robustness Analysis (RQ3). (a) Performance degradation under increasing input noise, CURVE remains robust while baselines collapse. (b-c) Uncertainty quantification. The density of uncertainty scores for correct vs. wrong predictions showing clear separation, which indicates the model is well-calibrated even under domain shift.