Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Individuation of 3D perceptual units from neurogeometry of binocular cells

Maria Virginia Bolelli, Giovanna Citti, Alessandro Sarti, Steven W. Zucker

TL;DR

A new framework for correspondence is introduced that integrates a neural-based algorithm to achieve stereo correspondence locally while, simultaneously, organizing the corresponding points into global perceptual units, resulting in an effective scene segmentation.

Abstract

We model the functional architecture of the early stages of three-dimensional vision by extending the neurogeometric sub-Riemannian model for stereo-vision introduced in \cite{BCSZ23}. A new framework for correspondence is introduced that integrates a neural-based algorithm to achieve stereo correspondence locally while, simultaneously, organizing the corresponding points into global perceptual units. The result is an effective scene segmentation. We achieve this using harmonic analysis on the sub-Riemannian structure and show, in a comparison against Riemannian distance, that the sub-Riemannian metric is central to the solution.

Individuation of 3D perceptual units from neurogeometry of binocular cells

TL;DR

A new framework for correspondence is introduced that integrates a neural-based algorithm to achieve stereo correspondence locally while, simultaneously, organizing the corresponding points into global perceptual units, resulting in an effective scene segmentation.

Abstract

We model the functional architecture of the early stages of three-dimensional vision by extending the neurogeometric sub-Riemannian model for stereo-vision introduced in \cite{BCSZ23}. A new framework for correspondence is introduced that integrates a neural-based algorithm to achieve stereo correspondence locally while, simultaneously, organizing the corresponding points into global perceptual units. The result is an effective scene segmentation. We achieve this using harmonic analysis on the sub-Riemannian structure and show, in a comparison against Riemannian distance, that the sub-Riemannian metric is central to the solution.
Paper Structure (34 sections, 5 theorems, 47 equations, 13 figures)

This paper contains 34 sections, 5 theorems, 47 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Background
  3. The stereo problem: state of the art and improvement with neurogeometry
  4. Neurogeometry of binocular vision and good continuation
  5. Connectivity kernel and mean field equation
  6. Stochastic paths and Fokker-Planck equation
  7. Time independent kernel
  8. Mean field equation and spectral analysis
  9. Existence and uniqueness of a solution
  10. Stability analysis
  11. Lyapunov method in the space of position and orientation.
  12. Perceptual units individuation and the correspondence problem
  13. Spectral analysis and emergence of perceptual units
  14. Spectral clustering, dimensionality reduction and salient objects in the visual scene
  15. Random walk normalization
  16. ...and 19 more sections

Key Result

Proposition 3.1

Let $\xi$ be a point in ${\mathbb {R}^3\rtimes\mathbb{S}^2}$ and $\rho_\text{}$ the fundamental solution of the operator $\partial_t + \mathcal{L}$ with $\mathcal{L}$ defined in fKoperator, and pole in $(\xi_0, 0)$, with $t_0 = 0$ without loss of generality. Then is fundamental solution for the operator $\mathcal{L}$:

Figures (13)

  • Figure 2.1: Display of connectivity. (a) Field Hayes and Hess association field (top) and 2D integral curves of the Citti-Sarti model CS06 (bottom). (b) General fan of integral curves described by equation \ref{['eq:integralCrv']} with varying $c_1$ and $c_2$ in ${\mathbb {R}}$, enveloping a curve $\gamma \in {\mathbb {R}}^3$. (c) Fan of 3D relatable points connected by integral curves \ref{['eq:integralCrv']}.
  • Figure 5.1: Display of $J_{{\mathbb {R}}^3}(\xi_j, \xi_0)$ for two different diffusion coefficients $\text{}$ (left $\text{} = 0.035$, right $\text{} = 0.08$), at isovalue $0.1$. Euler-Maruyama scheme parameters are $M = 400$, $T = 100$, $N = 10^6$.
  • Figure 5.2: Display of error bars on $J_\text{}^T(r_1)$ sections with $\text{} = 0.035$ and $T= 100$. The initial point $\xi_0$ is described by spatial indices $(r_{1_0},r_{2_0},r_{3_0})=(50, 1, 50)$, while $(\theta_0, \varphi_0)=(\pi/2, \pi/2)$. (a) Display of $J_\text{}^T (r_1)$ identified by $(\bar{r}_2, \bar{r}_3) = (10, 50)$ for two different numbers of paths $N$: blue color corresponds to $N= 10^6$, the black one to $N = 10^5$. Emphasized in the red square: the amplitude ($\Delta(N)$) of the two confidence intervals. The width of the blue interval is about half of the black one. (b) $J_\text{}^T (r_1)$ identified by $(\bar{r}_2, \bar{r}_3) = (50, 50)$ for two different number of paths: blue correspond to $N= 10^6$, black to $N = 10^5$. Moving away from the pole, both the kernel value and the confidence interval decrease.
  • Figure 5.3: Display of error bars and percentage error for $J_\text{}^T(r_2)$, with kernel parameters $\text{} = 0.035$ and $T= 100$, $N = 10^6$. The initial point $\xi_0$ is described by $(r_{1_0},r_{2_0},r_{3_0})=(50, 1, 50)$ and $(\theta_0, \varphi_0)=(\pi/2, \pi/2)$. The section has indices $(r_{1_0},r_{3_0})=(50, 50)$. (a) Display of error bars on $J_\text{}^T(r_2)$. (b) Display of the percentage error $E_{\%}=100\frac{r}{J_\text{}^T(r_2) }$, with $r = \frac{2.57 \bar{\sigma} }{\sqrt{N}}$.
  • Figure 5.4: Marginal projections on the plane $(r_1, r_2)$ to display dependence on the parameters $\text{}$ and $T$. The abscissa corresponds to $r_1$- axis, while the ordinate to $r_2$-axis. Columns describe the scale parameter $T \in \{ 15, 50, 100, 300, 600\}$, while rows correspond to different values of diffusion $\text{} \in \{ 0.01, 0.035\}$.
  • ...and 8 more figures

Theorems & Definitions (17)

  • Remark 3.1
  • Remark 3.2
  • Proposition 3.1
  • Remark 3.3
  • Remark 3.4: Symmetric kernel
  • Proposition 3.2
  • proof
  • Remark 3.5
  • Proposition 3.3
  • proof
  • ...and 7 more