Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Network Inpainting via Optimal Transport

Enrico Facca, Jan Martin Nordbotten, Erik Andreas Hanson

TL;DR

This work introduces Network Inpainting via Optimal Transport (NIOT), a model-driven framework for reconstructing network-like structures from corrupted images by coupling a data misfit with a physics-based regularizer inspired by branched transport. The regularizer is implemented as $\mathcal{R}(\mu)=\mathcal{E}(\mu)+\mathcal{M}_{\gamma}(\mu)$, where $\mu$ acts as a conductivity and $u(\mu)$ solves a PDE linking transport to the network path, enabling a continuous, topology-preserving prior. A two-pronged data fidelity approach maps conductivity to image space via either a simple identity $\mathcal{I}(\mu)=\alpha\mu$ or a porous-media-based map $\mathcal{I}(\mu)=\alpha\rho(t^*,m,\mu)$, with a weighted $L^{2}$ discrepancy $\mathcal{D}$ guiding the fit to observed data $I_{\text{obs}}$ under mask or full-domain confidence $W$. The method is realized on a Cartesian grid with gradient-based optimization and adjoint-enabled differentiation, demonstrates strong connectivity restoration and skeletonization in hand-drawn and frog-tongue networks, and shows potential for artifact pruning and data-informed reconstructions, albeit with nonconvex optimization and parameter-tuning challenges. The results indicate NIOT as a robust, physics-informed alternative to traditional inpainting for network-structured images, with applications spanning biology, hydrology, and image-based simulations.

Abstract

In this work, we present a novel tool for reconstructing networks from corrupted images. The reconstructed network is the result of a minimization problem that has a misfit term with respect to the observed data, and a physics-based regularizing term coming from the theory of optimal transport. Through a range of numerical tests, we demonstrate that our suggested approach can effectively rebuild the primary features of damaged networks, even when artifacts are present.

Network Inpainting via Optimal Transport

TL;DR

This work introduces Network Inpainting via Optimal Transport (NIOT), a model-driven framework for reconstructing network-like structures from corrupted images by coupling a data misfit with a physics-based regularizer inspired by branched transport. The regularizer is implemented as , where acts as a conductivity and solves a PDE linking transport to the network path, enabling a continuous, topology-preserving prior. A two-pronged data fidelity approach maps conductivity to image space via either a simple identity or a porous-media-based map , with a weighted discrepancy guiding the fit to observed data under mask or full-domain confidence . The method is realized on a Cartesian grid with gradient-based optimization and adjoint-enabled differentiation, demonstrates strong connectivity restoration and skeletonization in hand-drawn and frog-tongue networks, and shows potential for artifact pruning and data-informed reconstructions, albeit with nonconvex optimization and parameter-tuning challenges. The results indicate NIOT as a robust, physics-informed alternative to traditional inpainting for network-structured images, with applications spanning biology, hydrology, and image-based simulations.

Abstract

In this work, we present a novel tool for reconstructing networks from corrupted images. The reconstructed network is the result of a minimization problem that has a misfit term with respect to the observed data, and a physics-based regularizing term coming from the theory of optimal transport. Through a range of numerical tests, we demonstrate that our suggested approach can effectively rebuild the primary features of damaged networks, even when artifacts are present.
Paper Structure (22 sections, 1 theorem, 23 equations, 8 figures, 1 table)

This paper contains 22 sections, 1 theorem, 23 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Optimal and branched transport problems
  3. A reformulation of the BT
  4. Network inpainting via optimal transport
  5. Discrepancy term $\mathcal{D}$
  6. Mapping $\mu$ into $I$
  7. Discretization and optimization
  8. Software and implementation details
  9. Numerical experiments
  10. Y-shaped hand drawing
  11. Images with lost data (case $W=W_{\text{mask}}$)
  12. Images with lost data (case $W=1$)
  13. Influence of initial data $\mu_0$
  14. Images with artifacts
  15. Frog tongue
  16. ...and 7 more sections

Key Result

Lemma 1

Consider a graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, with nodes $\mathcal{V}$ and edges $\mathcal{E}$ that contains no loops (this type of graph is said to be acyclic). Fix an orientation on the edges $\mathcal{E}$ and let $\boldsymbol{f} \in \hbox{R}^{|\mathcal{V}|}$ whose entries sum up to ze where $\delta^{+}(k)$ and $\delta^{-}(k)$ denote the set of edge entering and existing the $k$-th n

Figures (8)

  • Figure 1: Schematic representation of the network reconstruction problem. In the upper panels, we report an image $I_{\text{true}}$ and a corrupted version ${I}_{\text{obs}}$. In the central panels, we report the images reconstructed using three inpainting algorithms implemented in ParSch2016, the harmonic, Cahn-Hilliard, and transport inpainting methods. In \ref{['fig:network-inpainting-btp']} we show the optimal network transporting $f^{+}$ to $f^{-}$. \ref{['fig:network-inpainting-rec']} reports a result obtained by the method presented in this paper taken from \ref{['fig:artifacts']}.
  • Figure 2: Spatial distribution of $DG^{0}$-approximation of the $\mu_{\text{opt},h}$ solving \ref{['prob:dmk']} for the $f^{+}$ and $f^{-}$ shown in \ref{['fig:network-inpainting-btp']} using $\gamma=0.8,0.5,0.2$ (top to bottom) and refining the initial $52\times 52$ grid $\mathcal{T} _{h}$(left to right)
  • Figure 3: Spatial distribution $\mu_{\text{rec},h}$ for different combinations of $\lambda$, maps, and confidence $W$. When the PM map is used, the blue line defines the boundary for which ${I}_{\text{rec}}=\mathcal{I}(\mu_{\text{rec},h})$ is $1e-3$. The red rectangle defines the mask region. The observed image ${I}_{\text{obs}}$ is reported in light gray.
  • Figure 4: Spatial distribution $\mu_{\text{rec},h}$ for $\lambda=1e-3,5e-2,1e-1,1e0$ (left to right) and $\mu_0=1,{\mu}_{\text{obs}}$ (top to bottom). Confidence $W=1$ and identity map. The observed data ${I}_{\text{obs}}$ is obtained by adding additional artifacts to the original network in \ref{['fig:network-inpainting-true']}, and then removing them within the mask (red rectangles). The result ${I}_{\text{obs}}$ is reported in light gray.
  • Figure 5: Frog tongue data. In \ref{['fig:frog-network-cohnheim']}, the original drawing from cohnheim1872investigations. In \ref{['fig:frog-network-opttdens']}, the spatial distribution of $\mu_{\text{opt},h}$ solving \ref{['prob:dmk']} for $f^{-}=1$ within the contour area and $f^{+}$ being the sum of two piecewise constant functions located at the root of the two main channels. In \ref{['fig:frog-network-shifted', 'fig:frog-network-mupou']}, the observed data ${I}_{\text{obs}}$ used in \ref{['sec:frog-shifted', 'sec:frog-mu']}, respectively. The circles indices the area where the mask is applied, the mask also used in \ref{['sec:frog-network']}.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Lemma 1
  • Remark 1