Beyond Single Images: Retrieval Self-Augmented Unsupervised Camouflaged Object Detection

TL;DR

CAMouflaged object detection faces difficulty distinguishing camouflaged targets from highly similar surroundings using only single-image cues. The paper introduces RISE, a retrieval self-augmented unsupervised COD framework that builds dataset-derived prototype libraries via Clustering-then-Retrieval and generates pseudo-labels with Multi-View KNN Retrieval. CR extracts high-quality foreground/background prototypes from the COD dataset through spectral clustering and cross-category retrieval with histogram-based filtering, while MVKR aggregates multi-view KNN votes to reduce artifact noise. Across COD benchmarks, RISE outperforms state-of-the-art unsupervised and prompt-based methods and offers substantial efficiency gains in pseudo-label generation, underscoring the value of dataset-level information for COD.

Abstract

At the core of Camouflaged Object Detection (COD) lies segmenting objects from their highly similar surroundings. Previous efforts navigate this challenge primarily through image-level modeling or annotation-based optimization. Despite advancing considerably, this commonplace practice hardly taps valuable dataset-level contextual information or relies on laborious annotations. In this paper, we propose RISE, a RetrIeval SElf-augmented paradigm that exploits the entire training dataset to generate pseudo-labels for single images, which could be used to train COD models. RISE begins by constructing prototype libraries for environments and camouflaged objects using training images (without ground truth), followed by K-Nearest Neighbor (KNN) retrieval to generate pseudo-masks for each image based on these libraries. It is important to recognize that using only training images without annotations exerts a pronounced challenge in crafting high-quality prototype libraries. In this light, we introduce a Clustering-then-Retrieval (CR) strategy, where coarse masks are first generated through clustering, facilitating subsequent histogram-based image filtering and cross-category retrieval to produce high-confidence prototypes. In the KNN retrieval stage, to alleviate the effect of artifacts in feature maps, we propose Multi-View KNN Retrieval (MVKR), which integrates retrieval results from diverse views to produce more robust and precise pseudo-masks. Extensive experiments demonstrate that RISE outperforms state-of-the-art unsupervised and prompt-based methods. Code is available at https://github.com/xiaohainku/RISE.

Figures (5)

• Figure 1: LEFT: t-SNE visualization of DINOv2 oquab2023dinov2 features and segmentation results from previous unsupervised methods. The highly similar characteristics between the camouflaged object and its environment cause the intra-image similarity-based approach to underperform on the COD task. RIGHT: Similarity distributions of global features of camouflaged objects and environments on the respective foreground and background prototype libraries. These prototype libraries are constructed from the COD dataset. Both the camouflaged object and the environment show a higher similarity to their corresponding prototype libraries, suggesting that dataset-level information can be effectively utilized to distinguish between the highly similar foreground and background in a single image.
• Figure 2: Overview of RISE. RISE is composed of two main stages: Clustering-then-Retrieval (CR) and KNN Retrieval. These stages work together to generate prototype libraries and retrieve camouflaged objects. In the CR phase, we start by clustering the feature maps from DINO for each image in the dataset, which generates a coarse mask. This mask allows us to extract both local and global features for the foreground and background. Using cross-category retrieval, we then retrieve high-quality prototypes from the local features. By aggregating the prototypes across all images, we create the final foreground and background prototype libraries. In the second stage, KNN retrieval, we apply KNN to identify the top-$K$ most similar prototypes in prototype libraries for each feature in the feature map. A voting mechanism is then used to classify each feature as either foreground or background.
• Figure 3: LEFT: Histogram of the global feature similarity distribution across all images in the dataset. In this study, we focus only on the similarity values to the left of the peak (indicated by the red dashed line). RIGHT: Multi-View KNN Retrieval. To mitigate noisy retrieval results (highlighted in the red box) caused by artifacts in the feature map, we combine the masks generated by transforming the image through different viewpoints, resulting in a final fused mask.
• Figure 4: Qualitative comparisons with four SOTA unsupervised methods. RISE* denotes RISE without Multi-View Retrieval.
• Figure 5: Left: Sensitivity analysis of the top-$K$ hyperparameter; Right: Impact of the dataset size.