Towards Size-invariant Salient Object Detection: A Generic Evaluation and Optimization Approach

TL;DR

This work tackles the sensitivity of SOD evaluation metrics to object size by formalizing a size-invariant framework, SIEva, that evaluates each salient region independently before aggregating results. It then operationalizes this principle into SIOpt, an optimization framework that per-objectizes losses (e.g., SI-BCE, SI-Dice, SI-AUC) and introduces PBAcc to make AUC-based, size-invariant optimization tractable. The authors provide a generalization bound for SIOpt, and extensive experiments show consistent improvements across RGB, RGB-D, RGB-T, and foundation-model settings, with pronounced gains for small or multi-object scenes. The approach yields model-agnostic benefits, improves robustness to size imbalance, and demonstrates practical scalability with modest overheads and memory efficiency when using PBAcc. Overall, SIOpt and SIEva offer a principled path to fairer, more reliable SOD in real-world, multi-object environments.$

Abstract

This paper investigates a fundamental yet underexplored issue in Salient Object Detection (SOD): the size-invariant property for evaluation protocols, particularly in scenarios when multiple salient objects of significantly different sizes appear within a single image. We first present a novel perspective to expose the inherent size sensitivity of existing widely used SOD metrics. Through careful theoretical derivations, we show that the evaluation outcome of an image under current SOD metrics can be essentially decomposed into a sum of several separable terms, with the contribution of each term being directly proportional to its corresponding region size. Consequently, the prediction errors would be dominated by the larger regions, while smaller yet potentially more semantically important objects are often overlooked, leading to biased performance assessments and practical degradation. To address this challenge, a generic Size-Invariant Evaluation (SIEva) framework is proposed. The core idea is to evaluate each separable component individually and then aggregate the results, thereby effectively mitigating the impact of size imbalance across objects. Building upon this, we further develop a dedicated optimization framework (SIOpt), which adheres to the size-invariant principle and significantly enhances the detection of salient objects across a broad range of sizes. Notably, SIOpt is model-agnostic and can be seamlessly integrated with a wide range of SOD backbones. Theoretically, we also present generalization analysis of SOD methods and provide evidence supporting the validity of our new evaluation protocols. Finally, comprehensive experiments speak to the efficacy of our proposed approach. The code is available at https://github.com/Ferry-Li/SI-SOD.

Figures (30)

• Figure 1: Statistics on dataset MSOD. \ref{['fig:msod_area']}: The $x$-axis indicates object size ranges, where Size(%) refers to the proportion of an object's area (i.e., each connected region as defined in Sec. \ref{['principles of SI_eval']}) relative to the total image area. Each positive integer $k$ on the $x$-axis represents the number of objects whose size ratio falls within the interval $[(k-1)\%, k\%]$. This figure highlights small objects ($1 \leq k \leq 10$) to clearly illustrate the motivation behind our study. \ref{['fig:msod_num']}: Depicts the average number of objects frequently present in each image, revealing that real-world SOD scenarios typically involve multiple salient objects.
• Figure 2: Illustration of the strengths of our proposed size-invariant metrics over conventional size-sensitive evaluations. Two representative examples are presented with $\mathsf{MAE}$ and $\mathsf{SI\text{-}MAE}$. Apparently, $\mathsf{SI\text{-}MAE}$ offers a more proper assessment of model performance in multiple SOD scenarios. More examples are deferred in \ref{['app_fig:eg']} in the Appendix.
• Figure 3: Examples of partitions. In \ref{['fig:object_frame1']}, there is a foreground frame ➀ and a background frame ➁. In \ref{['fig:object_frame2']}, there are five foreground frames from ➀ to ➄, and a background frame ➅.
• Figure 4: Statistics of the average intersection percentage across real-world SOD datasets, where the intersection percentage for each image is defined as the ratio of overlapping pixels to the total image size ($384 \times 384$). Detailed descriptions of these datasets can be found in \ref{['dataset_appendix']}.
• Figure 5: Qualitative visualizations on different backbones before and after using our proposed $\mathsf{SIOpt1}$.

Theorems & Definitions (21)

• Definition 3.1
• Definition 3.2
• Proposition 3.3
• Proposition 3.4
• Proposition 3.5
• Proposition 3.6
• Proposition 4.1
• Proposition 4.2
• Theorem 5.1: Generalization Bound for SI-SOD
• proof