S3OD: Towards Generalizable Salient Object Detection with Synthetic Data

TL;DR

S3OD addresses the data scarcity and ambiguity inherent in salient object detection by generating a large-scale synthetic dataset through a multi-modal diffusion pipeline and by adopting an ambiguity-aware, multi-mask decoder. The approach couples diffusion-derived features, concept attention, and self-supervised DINO-v3 representations to produce finely aligned image–mask pairs, and employs an iterative data-synthesis loop to target challenging categories. Empirical results show strong cross-dataset generalization from synthetic data alone, state-of-the-art performance after fine-tuning on DIS-5K and HR-SOD benchmarks, and clear ablations confirming the value of each component. The work demonstrates that scalable synthetic data, coupled with principled uncertainty handling, can substantially reduce annotation burden while enhancing generalization for dense prediction tasks.

Abstract

Salient object detection exemplifies data-bounded tasks where expensive pixel-precise annotations force separate model training for related subtasks like DIS and HR-SOD. We present a method that dramatically improves generalization through large-scale synthetic data generation and ambiguity-aware architecture. We introduce S3OD, a dataset of over 139,000 high-resolution images created through our multi-modal diffusion pipeline that extracts labels from diffusion and DINO-v3 features. The iterative generation framework prioritizes challenging categories based on model performance. We propose a streamlined multi-mask decoder that naturally handles the inherent ambiguity in salient object detection by predicting multiple valid interpretations. Models trained solely on synthetic data achieve 20-50% error reduction in cross-dataset generalization, while fine-tuned versions reach state-of-the-art performance across DIS and HR-SOD benchmarks.

Figures (8)

• Figure 1: S3OD Top: Our large scale synthetic dataset, consisting of diverse complex scenes and high quality samples. Bottom: Model Predictions. Our model trained on synthetic data generalizes well to real-world images, handling ambiguous scenes by predicting alternative hypothesis.
• Figure 2: S3OD Architecture. Model extends DPT dpt to predict multiple mask candidates and a vector of IoUs with the ground truth, employing DINO-v3 as the backbone. During training, the loss is propagated through the branch with the highest predicted IoU.
• Figure 3: Iterative Generation Pipeline. The LLM gpt generates a library of diverse prompts for a large set of object categories. These prompts guide a diffusion model to generate synthetic images with corresponding masks. The resulting dataset trains SOD model, which undergoes category-wise evaluation. Performance feedback from a trained SOD model dynamically adjusts category weights $\tilde{w_i}$, prioritizing challenging cases in subsequent iterations.
• Figure 4: S3OD Dataset: The dataset consists of diverse object categories and complex scenes that closely reflect real-world environments, featuring various lighting conditions, spatial compositions, and object interactions. All samples are generated with multi-modal dataset diffusion.
• Figure 5: The complete system prompt used to instruct the LLM gpt for generating diverse text descriptions. These descriptions focus on creating natural scenes with varying complexity, occlusion, and lighting to simulate challenging real-world conditions for salient object detection.