SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds

TL;DR

SoftPQ addresses brittleness in segmentation evaluation by introducing soft matching with tunable thresholds and a sublinear penalty, enabling a continuous score that differentiates near-misses from major errors. It generalizes PQ and recovers the original PQ when $h=l=0.5$, providing backward compatibility. Controlled experiments show smoother, more interpretable responses to perturbations and greater robustness to over- and under-segmentation, as well as favorable real-world performance (e.g., on Cellpose outputs). This yields actionable feedback for model development and benchmarking across diverse domains, supporting safer and more effective iterative refinement of segmentation models.

Abstract

Segmentation evaluation metrics traditionally rely on binary decision logic: predictions are either correct or incorrect, based on rigid IoU thresholds. Detection--based metrics such as F1 and mAP determine correctness at the object level using fixed overlap cutoffs, while overlap--based metrics like Intersection over Union (IoU) and Dice operate at the pixel level, often overlooking instance--level structure. Panoptic Quality (PQ) attempts to unify detection and segmentation assessment, but it remains dependent on hard-threshold matching--treating predictions below the threshold as entirely incorrect. This binary framing obscures important distinctions between qualitatively different errors and fails to reward gradual model improvements. We propose SoftPQ, a flexible and interpretable instance segmentation metric that redefines evaluation as a graded continuum rather than a binary classification. SoftPQ introduces tunable upper and lower IoU thresholds to define a partial matching region and applies a sublinear penalty function to ambiguous or fragmented predictions. These extensions allow SoftPQ to exhibit smoother score behavior, greater robustness to structural segmentation errors, and more informative feedback for model development and evaluation. Through controlled perturbation experiments, we show that SoftPQ captures meaningful differences in segmentation quality that existing metrics overlook, making it a practical and principled alternative for both benchmarking and iterative model refinement.

Figures (6)

• Figure 1: Limitations of Standard Segmentation Metrics in Handling over- and under-segmentation: Synthetic examples illustrating common segmentation errors—partial segmentation, over-segmentation, and ghost segments—for both single-object and multi-object cases. Despite there being qualitative differences in the predicted masks, traditional metrics (F1, mAP, IoU, PQ) often produce same or similar scores, highlighting their limited ability to distinguish between qualitatively distinct errors.
• Figure 2: Metric Sensitivity to Progressive Erosion: (Left) Metric scores as predicted masks are eroded over $25$ iterations, simulating increasing under-segmentation. (Right) Per-step score change ($\Delta$Score) highlights the stability and sensitivity of each metric. SoftPQ (red) shows a smooth, consistent decline, while F1, mAP, and PQ respond abruptly or inconsistently to minor perturbations. This illustrates SoftPQ's superior resolution in capturing gradual degradation.
• Figure 3: Sensitivity of SoftPQ to Lower IoU Thresholds: Left: Progressive dilation; Right: Progressive erosion of predicted masks. Each curve represents SoftPQ computed with a different lower IoU threshold $l$, while fixing the upper threshold $h=0.5$. As thresholds become more lenient, SoftPQ scores degrade more smoothly and maintain interpretability under noisy or degraded predictions. In contrast, PQ (black) drops sharply when predictions fall below its strict matching condition, failing to reflect gradual changes in quality.
• Figure 4: Robustness to over-segmentation: Impact of over-segmentation on various segmentation metrics. Each ground truth object is progressively split into 1–5 predicted segments, while maintaining a perfect 1:1 object-level match. SoftPQ (red) shows the most robust behavior, degrading gradually due to its tolerance for partial matches at low IoU thresholds ($l=0.05$). PQ (black) exhibits a more rigid drop, and mAP (blue) declines sharply due to penalizing redundant detections. The shaded region represents the envelope of SoftPQ scores across different $l$ values, indicating the flexibility of the metric under controlled segmentation fragmentation.
• Figure 5: Comparison between PQ and SoftPQ on the Cellpose dataset: (Left) SoftPQ scores plotted against PQ for each sample. SoftPQ is equal to or higher than PQ in all cases. (Right) The difference SoftPQ -- PQ plotted against PQ. Larger improvements from SoftPQ are observed in lower-PQ cases, often associated with over-segmentation or under-segmentation errors.