RUN: Reversible Unfolding Network for Concealed Object Segmentation

TL;DR

This work addresses concealed object segmentation (COS) by formulating COS as a foreground-background separation problem equipped with a residual sparsity constraint. It introduces RUN, a deep unfolding network that reversibly models foreground and background in both the mask and RGB domains, unfolding a proximal-gradient optimization into a multi-stage architecture where each stage contains Segmentation-Oriented Foreground Separation (SOFS) and Reconstruction-Oriented Background Extraction (ROBE). The framework Leveraging Reversible State Space (RSS) and Visual State Space (VSS) modules, together with an encoder-based semantic cue, enables non-local information capture and improved boundary refinement; an auxiliary edge stream further enhances segmentation. Extensive experiments across COD, PIS, MTOS, TOD, and related tasks demonstrate state-of-the-art performance and robust generalization, with RUN+ and refiner variants showing improved resilience to degradation. The results highlight the potential of unfolding-based approaches to balance interpretability and performance in high-level vision tasks beyond traditional low-level applications.

Abstract

Existing concealed object segmentation (COS) methods frequently utilize reversible strategies to address uncertain regions. However, these approaches are typically restricted to the mask domain, leaving the potential of the RGB domain underexplored. To address this, we propose the Reversible Unfolding Network (RUN), which applies reversible strategies across both mask and RGB domains through a theoretically grounded framework, enabling accurate segmentation. RUN first formulates a novel COS model by incorporating an extra residual sparsity constraint to minimize segmentation uncertainties. The iterative optimization steps of the proposed model are then unfolded into a multistage network, with each step corresponding to a stage. Each stage of RUN consists of two reversible modules: the Segmentation-Oriented Foreground Separation (SOFS) module and the Reconstruction-Oriented Background Extraction (ROBE) module. SOFS applies the reversible strategy at the mask level and introduces Reversible State Space to capture non-local information. ROBE extends this to the RGB domain, employing a reconstruction network to address conflicting foreground and background regions identified as distortion-prone areas, which arise from their separate estimation by independent modules. As the stages progress, RUN gradually facilitates reversible modeling of foreground and background in both the mask and RGB domains, directing the network's attention to uncertain regions and mitigating false-positive and false-negative results. Extensive experiments demonstrate the superior performance of RUN and highlight the potential of unfolding-based frameworks for COS and other high-level vision tasks. We will release the code and models.

Figures (8)

• Figure 1: Results of existing COS methods Our RUN demonstrates superiority in accurately segmenting concealed objects (in the top section) and achieves leading places across multiple COS tasks (in the bottom section): camouflaged object detection (COD), polyp image segmentation (PIS), medical tubular object segmentation (MTOS), and transparent object detection (TOD). In the top section, concealed object masks are highlighted in blue and pink, overlaid on the original images for visual clarity. FEDER+ and SINet+ indicate integrating FEDER and SINet with our RUN framework. For the bottom section, we employ commonly used datasets, methods, and metrics.
• Figure 2: Correspondence between uncertainties in the mask domain and distortions in the RGB domain. ${\mathbf{C}}$ is the concealed image and $\hat{\mathbf{B}}$ is the estimated background, which has conflicting judgments of concealed regions with the mask $\mathbf{M}$. This conflict leads to distortion-prone areas in their direct combination (g). Panel (h) illustrates the difference between (g) and the original image (a). However, after refinement through the network $\mathcal{B}(\mathbin{\vcenter{\hbox{$\m@th\bullet$}}})$, the reconstructed image $\hat{\mathbf{C}}$ becomes much closer to the original image, accompanied by a refined background $\mathbf{B}$ with improved accuracy. This refined background is passed to the next stage to further facilitate segmentation.
• Figure 3: Framework of our RUN. The network connections in $\hat{\mathcal{M}}(\cdot)$ and $\hat{\mathcal{B}}(\cdot)$ are derived strictly based on mathematical principles, thus enhancing interpretability. For clarity, we replace certain redundant details with $\mathbf{Q}_\mathbf{a}$, $\mathbf{Q}_\mathbf{b}$, and $\mathbf{Q}_\mathbf{c}$ and present $\hat{\mathcal{M}}(\cdot)$ according to \ref{['Eq:SOFSMhat']}. The detailed connection can be seen in \ref{['fig:SOFS_Details']} in the Appendix. Panel (ii) illustrates that the joint optimization of image segmentation and reconstruction tasks facilitates the network's progression toward an optimal solution.
• Figure 4: Visual comparison on COD, PIS, MTOS, and TOD tasks.
• Figure 5: Performance in degraded COS scenarios.