A Novel Distance-Based Metric for Quality Assessment in Image Segmentation

TL;DR

Problem: existing segmentation quality metrics either ignore spatial error distribution or yield unnormalized distance values that hinder cross-dataset comparisons. Approach: introduce the Surface Consistency Coefficient (SCC), defined as $\text{SCC}(S_{\text{pr}},S_{\text{gt}}) = \frac{1}{|E|}\sum_{x\in E} f(d_{\partial\text{gt}}(x))$ with a logistic weight $f_{\log}(r) = \frac{1}{1+\exp(-a(r-k))}$ and $d_{\partial\text{gt}}(x)$ the distance to the ground-truth surface, normalized to $[0,1]$. Findings: SCC differentiates proximal versus distal errors across synthetic 3D geometries and a concrete crack dataset, and outperforms traditional metrics in interpretability and cross-dataset comparison. Significance: SCC provides a practical, geometry-agnostic tool to assess segmentation quality, enabling targeted improvements and method selection depending on whether shape fidelity or surface detail matters.

Abstract

The assessment of segmentation quality plays a fundamental role in the development, optimization, and comparison of segmentation methods which are used in a wide range of applications. With few exceptions, quality assessment is performed using traditional metrics, which are based on counting the number of erroneous pixels but do not capture the spatial distribution of errors. Established distance-based metrics such as the average Hausdorff distance are difficult to interpret and compare for different methods and datasets. In this paper, we introduce the Surface Consistency Coefficient (SCC), a novel distance-based quality metric that quantifies the spatial distribution of errors based on their proximity to the surface of the structure. Through a rigorous analysis using synthetic data and real segmentation results, we demonstrate the robustness and effectiveness of SCC in distinguishing errors near the surface from those further away. At the same time, SCC is easy to interpret and comparable across different structural contexts.

Figures (7)

• Figure 1: (a) Data: section image of a 3D volume of overlapping spheres. (b-d,f-h) Weight maps using different parameter combinations for $f_{\log}$ (e). $a=1, k=5$ defines a suitable proximity range and transition speed for this geometry.
• Figure 2: Synthetic segmentations created by introducing systematic errors with 15% error rate visualized on a 256$^2$ section image from a 3D volume of overlapping spheres.
• Figure 3: Comparison of quality metrics for systematic errors on an image of overlapping cylinders of radius 10.5 pixels, height 210 pixels and volume density 50%. (a) sectional image of the structure. (b) histogram of distances from the surface split into foreground (negative) and background (positive) pixels. (c) largest distance of an error to the surface for proximate errors and smallest distance to the surface for distant errors. (e-f) Metrics for varying error rate.
• Figure 4: Comparison of quality metrics for systematic errors on an image of non-overlapping cubes with side length 30 pixels and volume density 10%. (a) sectional image of the structure. (b) histogram of distances from the surface split into foreground (negative) and background (positive) pixels. (c) largest distance of an error to the surface for proximate errors and smallest distance to the surface for distant errors. (e-f) Metrics for varying error rate.
• Figure 5: Evaluation of SCC on an image of overlapping spheres with radius 20 and volume density 30% (a). (b)-(d) and (f)-(g) results for varying parameters $a$ and $k$. (e) Largest distance of an error to the surface for proximate errors and the smallest distance for distant errors. See Figure \ref{['fig:AllGeometries']} (e) for legend.