UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification

TL;DR

UGG-ReID tackles robust multi-modal object ReID by explicitly modeling both fine-grained local uncertainty and sample-level uncertainty. It combines a Gaussian Patch-Graph Representation (GPGR) for uncertainty-aware local-global feature modeling with an Uncertainty-Guided Mixture of Experts (UGMoE) to route samples to low-uncertainty experts and strengthen cross-modal interactions. The method achieves state-of-the-art results across five datasets, exhibits strong noise immunity, and runs with high efficiency (about 371 FPS on a single GPU). These contributions offer a principled, scalable approach to robust multi-modal fusion under realistic perceptual disturbances.

Abstract

Multi-modal object Re-IDentification (ReID) has gained considerable attention with the goal of retrieving specific targets across cameras using heterogeneous visual data sources. At present, multi-modal object ReID faces two core challenges: (1) learning robust features under fine-grained local noise caused by occlusion, frame loss, and other disruptions; and (2) effectively integrating heterogeneous modalities to enhance multi-modal representation. To address the above challenges, we propose a robust approach named Uncertainty-Guided Graph model for multi-modal object ReID (UGG-ReID). UGG-ReID is designed to mitigate noise interference and facilitate effective multi-modal fusion by estimating both local and sample-level aleatoric uncertainty and explicitly modeling their dependencies. Specifically, we first propose the Gaussian patch-graph representation model that leverages uncertainty to quantify fine-grained local cues and capture their structural relationships. This process boosts the expressiveness of modal-specific information, ensuring that the generated embeddings are both more informative and robust. Subsequently, we design an uncertainty-guided mixture of experts strategy that dynamically routes samples to experts exhibiting low uncertainty. This strategy effectively suppresses noise-induced instability, leading to enhanced robustness. Meanwhile, we design an uncertainty-guided routing to strengthen the multi-modal interaction, improving the performance. UGG-ReID is comprehensively evaluated on five representative multi-modal object ReID datasets, encompassing diverse spectral modalities. Experimental results show that the proposed method achieves excellent performance on all datasets and is significantly better than current methods in terms of noise immunity. Our code is available at https://github.com/wanxixi11/UGG-ReID.

Figures (8)

• Figure 1: Modeling aleatoric uncertainty in multi-modal ReID. (a) The challenges in multi-modal object ReID. (b) Local uncertainty modeling for modality-specific refinement. (c) Joint uncertainty modeling in samples and modalities for improved fusion.
• Figure 2: The overall framework of the proposed Uncertainty-Guided multi-modal object ReID (UGG-ReID), which is composed of two main components: Gaussian Patch-Graph Representation (GPGR) and Uncertainty-Guided Mixture of Experts (UGMoE).
• Figure 3: Robustness analysis on RGBNT201. Evaluation results with (a) different levels of Gaussian noise added during dataset generation, and (b) varying noise intensities added during testing after training on clean data.
• Figure 4: Retrieval results under different testing conditions after training on clean data. (a) Clean. (b) Gaussian noise. (c) Arbitrary noise. Green/Red boxes indicate correct/incorrect retrieval results.
• Figure 5: Visualization results of the (a) Input image. (b) Baseline. (c) UGG-ReID (Ours).