Causal-Driven Feature Evaluation for Cross-Domain Image Classification
Chen Cheng, Ang Li
TL;DR
The paper tackles the challenge of out-of-distribution generalization by arguing that domain-invariant features are not necessarily causally useful for prediction. It introduces a causal evaluation framework based on the probability of necessity and sufficiency ($PNS$) applied at the level of latent segments, requiring semantically aligned latent coordinates across domains. A two-stage approach learns a shared latent structure and then uses Top-$K$ segment selection guided by cross-domain $PNS$ scores to build a robust OOD classifier. The method is validated on controlled multi-domain MNIST and the PACS benchmark, showing consistent improvements under strong domain shifts and demonstrating the practical value of explicit causal evaluation for robust generalization.
Abstract
Out-of-distribution (OOD) generalization remains a fundamental challenge in real-world classification, where test distributions often differ substantially from training data. Most existing approaches pursue domain-invariant representations, implicitly assuming that invariance implies reliability. However, features that are invariant across domains are not necessarily causally effective for prediction. In this work, we revisit OOD classification from a causal perspective and propose to evaluate learned representations based on their necessity and sufficiency under distribution shift. We introduce an explicit segment-level framework that directly measures causal effectiveness across domains, providing a more faithful criterion than invariance alone. Experiments on multi-domain benchmarks demonstrate consistent improvements in OOD performance, particularly under challenging domain shifts, highlighting the value of causal evaluation for robust generalization.