DeepBranchTracer: A Generally-Applicable Approach to Curvilinear Structure Reconstruction Using Multi-Feature Learning

TL;DR

DeepBranchTracer tackles the challenge of reconstructing curvilinear structures from images by integrating external image cues with internal geometric information through a unified curvilinear structure feature learning network (CSFL) and a multi-feature fusion tracing strategy (MFT). The method reframes reconstruction as geometric parameter estimation, learning centerline and boundary from images while simultaneously predicting radius and direction via a sequential (LSTM) module, and iteratively tracing branches without relying on handcrafted parametric models. Key contributions include the CSFL with four blocks (centerline, boundary, direction, radius), a geometry-plus-image loss framework, and the MFT tracing procedure that refines positions using both feature types. Across 2D road and vessel datasets and 3D neuron datasets, the approach achieves superior continuity and accuracy, demonstrating strong generalization and applicability to diverse curvilinear structures without domain-specific geometry priors.

Abstract

Curvilinear structures, which include line-like continuous objects, are fundamental geometrical elements in image-based applications. Reconstructing these structures from images constitutes a pivotal research area in computer vision. However, the complex topology and ambiguous image evidence render this process a challenging task. In this paper, we introduce DeepBranchTracer, a novel method that learns both external image features and internal geometric characteristics to reconstruct curvilinear structures. Firstly, we formulate the curvilinear structures extraction as a geometric attribute estimation problem. Then, a curvilinear structure feature learning network is designed to extract essential branch attributes, including the image features of centerline and boundary, and the geometric features of direction and radius. Finally, utilizing a multi-feature fusion tracing strategy, our model iteratively traces the entire branch by integrating the extracted image and geometric features. We extensively evaluated our model on both 2D and 3D datasets, demonstrating its superior performance over existing segmentation and reconstruction methods in terms of accuracy and continuity.

Figures (6)

• Figure 1: Motivation. Segmentation methods focus on learning image features but tend to misinterpret two adjacent branches as an entity. Reconstruction methods capture the geometric features of branches but rely on complex parameterized models. The integration of these two features offers the potential to improve the continuity of results.
• Figure 2: Architecture of the CSFL network. The network integrates four sub-task blocks in a U-shaped network to extract both geometric features (including radius and direction blocks) and image features (including centerline and boundary blocks).
• Figure 3: MFT strategy. Step 1: Find an initial point $\tilde{\mathbf{c}}(t_i)$ based on the geometric features of direction and radius. Step 2: From the point set $\mathbf{Q}$ of the surrounding centerline, locate the most potential point $\hat{\mathbf{c}}(t_i)$ to adjust the position. Step 3: Determine whether to trace the successor point based on the boundary probability $\hat{y}_b(\hat{\mathbf{c}}(t_{i}))$.
• Figure 4: From left to right, with the integration of centerline and geometric features, the reconstruction branch of M5 achieves improved smoothness and continuity. The yellow dots represent the termination points of the broken branches.
• Figure 5: Reconstruction results of road and vessel datasets. The yellow arrows indicate our model detects the road which is missing from the ground truth label. The green arrows indicate the branches are always broken here.