Riemannian Patch Assignment Gradient Flows

TL;DR

The paper addresses graph-based label assignment with the need to jointly regularize spatial structure and label interactions.It introduces patch assignment flows on a patch assignment manifold, using a dictionary of labeled templates and a patch dictionary graph to encode nonlocal consistency, and optimizes a Riemannian gradient flow derived from a Lagrangian action functional.Key contributions include the formal definition of P-AF, orientation independence, an action-functional characterization, and demonstrations of symmetry-based uncertainty quantification and real-data applicability.This framework enables principled nonlocal regularization of graph labelings via learned patch interactions, with potential for dictionary-driven design and uncertainty-aware predictions.

Abstract

This paper introduces patch assignment flows for metric data labeling on graphs. Labelings are determined by regularizing initial local labelings through the dynamic interaction of both labels and label assignments across the graph, entirely encoded by a dictionary of competing labeled patches and mediated by patch assignment variables. Maximal consistency of patch assignments is achieved by geometric numerical integration of a Riemannian ascent flow, as critical point of a Lagrangian action functional. Experiments illustrate properties of the approach, including uncertainty quantification of label assignments.

Figures (6)

• Figure 1: (a) Directed grid graph $\mathcal{G}_{\mathcal{V}}$ with an arbitrary orientation. (b) Intersection supports $[i]_{\mathcal{V}}$ and $[j]_{\mathcal{V}}$ of patches centered at an horizontal edge. (c) A small patch dictionary $\mathcal{D}$ with 16 binary labeled patch templates. (d) Few patch template adjacency relations based on \ref{['eq:om-patch-binary']}. The first two examples show horizontal adjacency relations and the last two vertical ones. For example, regarding the first graph, $7\to 10$ means moving from patch template 7 one vertex to the right (or equivalently, shifting patch template 7 one vertex to the left) may result in patch template 10, whereas shifting patch template 10 to the left may result in either patch template 1 or patch template 4. A loop at vertex $d$ indicates that patch template $d$ is self-adjacent. (e) The entire resulting patch dictionary graph $\mathcal{G}_{\mathcal{D}}$\ref{['eq:def-G-mcD']} comprising both horizontal and vertical patch template adjacencies.
• Figure 2: (a) A small dictionary of $5\times 5$ binary template patches, complemented by the constant background patch, which slightly extends the scenario of Figure \ref{['fig:patch-adjacency']}, yet still models a 'small world of binary crossing structure'. (b), (c) Patch template adjacency matrices \ref{['eq:def-Om-G-D']} for horizontal and vertical edges, respectively. (d) The resulting dictionary dictionary graph is highly symmetric. The 10 line patches (two leftmost columns in (a)) and the constant patch (not shown in (a)) are self-adjacent and correspond to loops in (d).
• Figure 3: (a) Input data, to be regularized by the patch-AF using the dictionary of Fig. \ref{['fig:dictionary-lines-5x5']}. (b)-(d) Labelings returned by the patch-AF when background and foreground labels of dictionary patches are equally important (ratio $1.0/1.0$; (b)) or not (ratio $1.0/1.2$ (c); ratio $1.0/1.5$ (d)). (e)--(g) Uncertainty quantification of the labelings above, due to the symmetry of the patch dictionary graph and the corresponding multiplicity of locally consistent labelings (see text). The colors 'red' and 'blue' signal unique fore- and background labelings, respectively, whereas 'white' signals uncertainty and plausible alternative labelings. This result illustrates that patch assignment flows enable both labeling pattern suppression and labeling pattern formation.
• Figure 4: (a) A light microscopy image of cross-sectional skeletal muscle structure with immunohistochemical staining (see text). (b) Raw input data (section) after histogram equalization (cf. also Figure \ref{['fig:MyoST-results']}(e)). (c) Few interrogation regions with raw data from the foreground class 1 (left), class 2 and the background class (top and right), used to train a SVM for local labeling $W^{0}$ of the entire image, which defines the initial point of the patch assignment flow by \ref{['eq:def-PO']}.
• Figure 5: (a) A dictionary of $5\times 5$labeled patches with foreground labels (black and white) and background labels (gray), complemented by the three constant patches (not shown). (b) The patch adjacency graph is structured so as to favour transitions from each foreground class to itself or to the background, respectively, rather than direct transitions between both foreground classes. The three constant patches are represented by the center vertex (background label) and the two extreme vertices (foreground labels), respectively. This structure of the patch dictionary graph encodes the prior knowledge that spatially connected components of both foreground regions should be separated by the background region.

Theorems & Definitions (3)

• Lemma 3.1: maximizing patch consistency
• Proposition 3.2
• Proposition 3.3: P-AF: action functional, stationary point