Representing Topological Self-Similarity Using Fractal Feature Maps for Accurate Segmentation of Tubular Structures

TL;DR

This work tackles the challenging segmentation of tubular structures by leveraging topological self similarity via fractal feature maps (FFMs). It extends fractal dimension to pixel level using a box counting approach and feeds the resulting $FFM_{image}$ into a multi decoder network (MD-Net) that also predicts edges and skeletons, with a fractal feature constrained loss weighting the object predictions by $FFM_{label}$. MD-Net outperforms strong baselines across five tubular datasets and a non tubular one, while FFMs also improve vanilla U-Net and HR-Net when used as plug‑ins. The combination yields improved boundary accuracy and topology preservation, with broad applicability and code available at the project repository.

Abstract

Accurate segmentation of long and thin tubular structures is required in a wide variety of areas such as biology, medicine, and remote sensing. The complex topology and geometry of such structures often pose significant technical challenges. A fundamental property of such structures is their topological self-similarity, which can be quantified by fractal features such as fractal dimension (FD). In this study, we incorporate fractal features into a deep learning model by extending FD to the pixel-level using a sliding window technique. The resulting fractal feature maps (FFMs) are then incorporated as additional input to the model and additional weight in the loss function to enhance segmentation performance by utilizing the topological self-similarity. Moreover, we extend the U-Net architecture by incorporating an edge decoder and a skeleton decoder to improve boundary accuracy and skeletal continuity of segmentation, respectively. Extensive experiments on five tubular structure datasets validate the effectiveness and robustness of our approach. Furthermore, the integration of FFMs with other popular segmentation models such as HR-Net also yields performance enhancement, suggesting FFM can be incorporated as a plug-in module with different model architectures. Code and data are openly accessible at https://github.com/cbmi-group/FFM-Multi-Decoder-Network.

Figures (5)

• Figure 1: Topological self-similarity in tubular structures. If we consider "one junction with multiple edges" as a basic component, complex tubular structures have similar components at different scales. The images on the right are magnified views of the rectangular regions on the left.
• Figure 2: Workflow of computing FFM of an image.
• Figure 3: Overview and details of our proposed model MD-Net. The left part illustrates the overall structure of MD-Net. (Image, $FFM_{image}$) is processed as input by the Encoder, which extracts features of varying sizes from the input. These features are transmitted simultaneously to three Decoders through Skip Connection. The right part is a detailed description of the Encoder and Decoders. Each Decoder performs upsampling and concatenation operations similar to U-Net to obtain the predictions, comprising the edge, skeleton, and object.
• Figure 4: Comparison of segmentation results. (a) Image. (b) Label. (c) Results of U-Net. (d) Results of existing SOTA approaches. From top to bottom, it's AF-Net, AF-Net, GT-DLA, AF-Net, and Dconn-Net. (e) Results of U-Net*. (f) Results of MD-Net*. Red: true positive. Green: false negative. Blue: false positive. In the ER, MITO, ROSE, STAR, and ROAD rows, columns (b) through (f) display the magnified results of the regions demarcated by yellow squares in the corresponding images of column (a).
• Figure : Generation of FFM.