High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinement

TL;DR

This work tackles high-quality Unknown Object Instance Segmentation (UOIS) in cluttered environments by introducing QuBER, a refinement framework that uses quadruple boundary error estimation (TP, TN, FP, FN) and an Error Guidance Fusion (EGF) module to correct both fine-grained and instance-level errors. The model integrates an IS feature extractor, an error estimator, and an error-informed refiner in an end-to-end architecture, sharing RGB-D instance features to maintain fast inference times (under 0.1 seconds). Training relies on synthetic perturbations of ground-truth masks to simulate realistic errors, with a composite loss that balances segmentation and error estimation objectives. Extensive experiments on OCID, OSD, and WISDOM demonstrate consistent improvements over state-of-the-art refinement methods across diverse initial segmentations, and robot experiments show higher target grasping success in cluttered bins. QuBER thus delivers practical, fast, and accurate UOIS refinements with tangible benefits for robotic manipulation in real-world scenes.

Abstract

Accurate and efficient segmentation of unknown objects in unstructured environments is essential for robotic manipulation. Unknown Object Instance Segmentation (UOIS), which aims to identify all objects in unknown categories and backgrounds, has become a key capability for various robotic tasks. However, existing methods struggle with over-segmentation and under-segmentation, leading to failures in manipulation tasks such as grasping. To address these challenges, we propose QuBER (Quadruple Boundary Error Refinement), a novel error-informed refinement approach for high-quality UOIS. QuBER first estimates quadruple boundary errors-true positive, true negative, false positive, and false negative pixels-at the instance boundaries of the initial segmentation. It then refines the segmentation using an error-guided fusion mechanism, effectively correcting both fine-grained and instance-level segmentation errors. Extensive evaluations on three public benchmarks demonstrate that QuBER outperforms state-of-the-art methods and consistently improves various UOIS methods while maintaining a fast inference time of less than 0.1 seconds. Furthermore, we show that QuBER improves the success rate of grasping target objects in cluttered environments. Code and supplementary materials are available at https://sites.google.com/view/uois-quber.

Figures (7)

• Figure 1: (Top) Results of the proposed QuBER method for high-quality UOIS across various domains. (Bottom) From the initial segmentation, QuBER performs error-informed refinement by estimating pixel-wise quadruple boundary errors and refining the segmentation based on these error estimates.
• Figure 2: Comparison of binary mask errors (True, False, shown in (e)) and our quadruple boundary errors (TP, TN, FP, FN, shown in (f) and (g)) to represent segmentation errors in the initial segmentation (b) for error estimation. The proposed quadruple boundary error estimation effectively captures instance-level errors and facilitates precise refined segmentation (c).
• Figure 3: Overview of QuBER for error-informed refinement.
• Figure 4: Examples of (a) RGB images, (b) ground truth (GT) masks, and (c) perturbed masks used during training. (d-f) the instance representations utilized in QuBER.
• Figure 5: High-quality UOIS results of QuBER on diverse scenes demonstrating accurate object instance segmentation