A Holistically Point-guided Text Framework for Weakly-Supervised Camouflaged Object Detection

TL;DR

This work tackles weakly-supervised camouflaged object detection (WSCOD) by introducing a holistically point-guided text framework that combines point and text supervision via SAM, Grounding DINO, and CLIP. The method segments candidate masks with a Point-guided Candidate Generation (PCG), selects the most text-aligned mask with a Qualified Candidate Discriminator (QCD), and then trains a self-supervised DINO ViT model using the selected pseudo mask. It also introduces two new datasets, P2C-COD and T-COD, and demonstrates large gains on CAMO, CHAMELEON, COD10K, and NC4K, even surpassing some fully-supervised COD methods. The results indicate the effectiveness of cross-modal prompting and coarse-to-fine pseudo-label generation for improving weakly-supervised COD, with potential for end-to-end future work.

Abstract

Weakly-Supervised Camouflaged Object Detection (WSCOD) has gained popularity for its promise to train models with weak labels to segment objects that visually blend into their surroundings. Recently, some methods using sparsely-annotated supervision shown promising results through scribbling in WSCOD, while point-text supervision remains underexplored. Hence, this paper introduces a novel holistically point-guided text framework for WSCOD by decomposing into three phases: segment, choose, train. Specifically, we propose Point-guided Candidate Generation (PCG), where the point's foreground serves as a correction for the text path to explicitly correct and rejuvenate the loss detection object during the mask generation process (SEGMENT). We also introduce a Qualified Candidate Discriminator (QCD) to choose the optimal mask from a given text prompt using CLIP (CHOOSE), and employ the chosen pseudo mask for training with a self-supervised Vision Transformer (TRAIN). Additionally, we developed a new point-supervised dataset (P2C-COD) and a text-supervised dataset (T-COD). Comprehensive experiments on four benchmark datasets demonstrate our method outperforms state-of-the-art methods by a large margin, and also outperforms some existing fully-supervised camouflaged object detection methods.

Figures (9)

• Figure 1: Illustration of our Holistically Point-guided Text framework for COD task, highlighting how the text path encounters various difficulties in identifying camouflaged objects. However, by leveraging point guidance, we can precisely correct or recover the loss detection of objects.
• Figure 2: Our complete framework consists of three phases: SEGMENT, CHOOSE, TRAIN for Weakly-supervised Camouflaged Object Detection
• Figure 3: Point-guided Candidate Generation (PCG): Given point and text prompts, we send the prompts into the SAM SegmentAnything-SAM to generate segmentation masks, respectively. In the text path, we first extract the bounding box, then apply point-guided bounding box correction, then determining the eligible mask through mask erasure for the object.
• Figure 4: Qualified Candidate Discriminator (QCD): Detailed illustration of choosing the qualified mask as the final pseudo label for training.
• Figure 5: The framework of our training network using a self-supervised DINO transformer.