Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Second-order difference subspace

Kazuhiro Fukui, Pedro H. V. Valois, Lincon Souza, Takumi Kobayashi

TL;DR

The second-order difference subspace is proposed, a higher-order extension of the first-order difference subspace between two subspaces that can analyze the geometrical difference between them and demonstrate the validity and naturalness of it by showing numerical results on two applications.

Abstract

Subspace representation is a fundamental technique in various fields of machine learning. Analyzing a geometrical relationship among multiple subspaces is essential for understanding subspace series' temporal and/or spatial dynamics. This paper proposes the second-order difference subspace, a higher-order extension of the first-order difference subspace between two subspaces that can analyze the geometrical difference between them. As a preliminary for that, we extend the definition of the first-order difference subspace to the more general setting that two subspaces with different dimensions have an intersection. We then define the second-order difference subspace by combining the concept of first-order difference subspace and principal component subspace (Karcher mean) between two subspaces, motivated by the second-order central difference method. We can understand that the first/second-order difference subspaces correspond to the velocity and acceleration of subspace dynamics from the viewpoint of a geodesic on a Grassmann manifold. We demonstrate the validity and naturalness of our second-order difference subspace by showing numerical results on two applications: temporal shape analysis of a 3D object and time series analysis of a biometric signal.

Second-order difference subspace

TL;DR

The second-order difference subspace is proposed, a higher-order extension of the first-order difference subspace between two subspaces that can analyze the geometrical difference between them and demonstrate the validity and naturalness of it by showing numerical results on two applications.

Abstract

Subspace representation is a fundamental technique in various fields of machine learning. Analyzing a geometrical relationship among multiple subspaces is essential for understanding subspace series' temporal and/or spatial dynamics. This paper proposes the second-order difference subspace, a higher-order extension of the first-order difference subspace between two subspaces that can analyze the geometrical difference between them. As a preliminary for that, we extend the definition of the first-order difference subspace to the more general setting that two subspaces with different dimensions have an intersection. We then define the second-order difference subspace by combining the concept of first-order difference subspace and principal component subspace (Karcher mean) between two subspaces, motivated by the second-order central difference method. We can understand that the first/second-order difference subspaces correspond to the velocity and acceleration of subspace dynamics from the viewpoint of a geodesic on a Grassmann manifold. We demonstrate the validity and naturalness of our second-order difference subspace by showing numerical results on two applications: temporal shape analysis of a 3D object and time series analysis of a biometric signal.
Paper Structure (19 sections, 8 theorems, 53 equations, 11 figures)

This paper contains 19 sections, 8 theorems, 53 equations, 11 figures.

Table of Contents

  1. Introduction
  2. First-order difference subspace
  3. Definitions in the simple case
  4. Definitions in general case
  5. Magnitude of difference subspace
  6. Second-order difference subspace
  7. Definition based on second-order central difference
  8. Mechanism of second-order DS
  9. Representation based on Grassman manifold
  10. Orthogonal decomposition of magnitude
  11. Numerical experiments
  12. Temporal analysis of deforming 3D shape
  13. Experimental setting
  14. Experimental results
  15. Time series analysis of biometric signal
  16. ...and 4 more sections

Key Result

Lemma 2.1

The orthonormal basis ${\mathbf{\Phi}} \in {\mathbb{R}}^{{n}\times{d_1}}$ of ${\mathcal{S}_1}$ and $\mathbf{\Psi} \in {\mathbb{R}}^{{n}\times{d_2}}$ of ${\mathcal{S}_2}$ ($d_1 \leq d_2$) can be orthogonally transformed to the following orthonormal bases ${\mathbf{\Phi}}^{*} \in {\mathbb{R}}^{{n}\tim where $\{{\boldsymbol{\gamma}}_i\}$ represents the orthonormal basis of an intersection subspace $\

Figures (11)

  • Figure 1: First-order difference subspace $\mathcal{D}$ between subspaces $\mathcal{S}_1$ and $\mathcal{S}_2$tpami2015.
  • Figure 2: Orthogonal decomposition of sum subspace $\mathcal{W}$ into three subspaces.
  • Figure 3: Three sequential vectors/subspaces in $n$-dimensional vector space.
  • Figure 4: Temporal sequential subspaces, $\mathcal{S}_1$, $\mathcal{S}_2$ and $\mathcal{S}_3$ on Grassman manifold $\mathsf{Gr}(d,n)$. (a) Zero second-order DS $\mathcal{D}^2$ in the case that two subspaces ${\mathcal{S}}_2$ and ${\mathcal{M}}$ coincide completely. (b) Second-order DS $\mathcal{D}^2$ in the case that ${\mathcal{S}}_2$ is slightly shifted from ${\mathcal{M}}$ along the geodesic. (c) Second-order DS $\mathcal{D}^2$ in that ${\mathcal{S}}_2$ is not on the geodesic.
  • Figure 5: Geometrical relationship of the related subspaces on the $\mathsf{Gr}(d,n)$ and the tangent space $T_M$ of $\mathsf{Gr}(d,n)$.
  • ...and 6 more figures

Theorems & Definitions (23)

  • Definition 2.1: Difference subspace $\mathcal{D}$
  • Definition 2.2: Principal component subspace (Karcher mean) $\mathcal{M}$
  • Lemma 2.1
  • Definition 2.3: Difference subspace $\mathcal{D}$
  • Definition 2.4: Principal component subspace $\mathcal{M}$
  • Definition 2.5: Intersection subspace $\mathcal{I}$
  • Definition 2.6: Magnitude of difference subspace
  • Definition 3.1: Second-order difference subspace
  • Definition 3.2: Magnitude of second-order difference subspace
  • Lemma 4.1
  • ...and 13 more