Higher order PCA-like rotation-invariant features for detailed shape descriptors modulo rotation

TL;DR

The paper proposes a PCA-like framework for rotation-invariant shape descriptors that extends from the classical second-order covariance to higher-order tensors and Gaussian-weighted polynomials, enabling detailed, continuous shape descriptions. Shapes are represented via expectations $E[f(x)]$, with a baseline covariance tensor $[p]$ defined by $p_{ab}=E[(x_a-E[x_a])(x_b-E[x_b])]$, and extended to order-3 moments $p_{abc}=E[(x_a-E[x_a])(x_b-E[x_b])(x_c-E[x_c])]$, along with a decodable density representation using a polynomial multiplied by a Gaussian and Hermite-polynomial bases. Rotation invariants are built through contractions and graph-based constructions, including invariants like $\mathrm{Tr}([p]^i)$ and Frobenius-type inner products, while recognizing the need for a complete, independent set of invariants in higher orders. To avoid expensive rotation optimization, the paper advocates using invariant feature vectors to quantify similarity modulo rotation, with extensions to handle shape variability via distributions of invariants across dynamics. The work lays a foundation for rotation-invariant shape descriptors applicable to molecular shapes and 2D/3D recognition, while outlining important open questions and practical directions for future research.

Abstract

PCA can be used for rotation invariant features, describing a shape with its $p_{ab}=E[(x_i-E[x_a])(x_b-E[x_b])]$ covariance matrix approximating shape by ellipsoid, allowing for rotation invariants like its traces of powers. However, real shapes are usually much more complicated, hence there is proposed its extension to e.g. $p_{abc}=E[(x_a-E[x_a])(x_b-E[x_b])(x_c-E[x_c])]$ order-3 or higher tensors describing central moments, or polynomial times Gaussian allowing decodable shape descriptors of arbitrarily high accuracy, and their analogous rotation invariants. Its practical applications could be rotation-invariant features to include shape modulo rotation e.g. for molecular shape descriptors, or for up to rotation object recognition in 2D images/3D scans, or shape similarity metric allowing their inexpensive comparison (modulo rotation) without costly optimization over rotations.

Figures (4)

• Figure 1: Having some shape/density, we can average ($E[]$) over it defining 2nd order polynomial with $[p]_{ab}\equiv p_{ab}=E[(x_a-E[x_a])(x_b-E[x_b])]$ covariance matrix, approximating this shape as ellipsoid. Finding such matrix for two shapes as $[p]$ and $[q]$, we can compare their eigenvalues/eigenvectors to find relative rotations/shapes with PCA-like approximation. To test differing by only rotation, we could check equality of $\forall_{i=1}^d \textrm{Tr}\left([p]^m\right) =\textrm{Tr}\left([q]^i\right)$ in $\mathbb{R}^d$ rotation invariants, agreement of all ensures similarity: $[p]\sim [q] \equiv \exists_{O\in \textrm{SO}(d)}: [p]=O[q]O^T$. As such ellipsoid approximation is insufficient to describe more complex shapes, here we propose to extend it to higher central moments, starting with $p_{abc}=E[(x_a-E[x_a])(x_b-E[x_b])(x_c-E[x_c])]$ order-3 tensor, for which we can extend $\textrm{Tr}([p]^i) =\textrm{Tr}([q]^i)$ rotation invariants with analogous graph-based invariants, starting with $\sum_{abc} (p_{abc})^2$. Bottom: averaged 28x28 MNIST mnist digits and such their PCA approximation.
• Figure 2: Visualization of accuracy of Gaussian times polynomial representation for MNIST mnist averaged digits as $d=2$ dimensional $28\times 28$ grayscale images, using $j_1+j_2=r$ for $r=0,..,m$ sums of polynomial degrees. Specifically, the grayness was normalized into density $\rho$ summing to 1 defining $E[]$, then center $(E[x_1], E[x_2])$ was subtracted from positions, then covariance matrix $[p]$ was calculated and coordinates were divided by $\sqrt{\textrm{Tr}([p])/d}$ for scale normalization. Then coordinates in product basis of (\ref{['herm']}) for $j_1+j_2\leq m$ were calculated, and used to reconstruct the images - which are shown. Due to replacing integration with summation over the lattice, the basis slightly loses orthonormality, what can be improved by Gram-Schmidt orthogonalization, at cost of less accurate representation.
• Figure 3: Diagrammatic representations (pnppoly) of some first rotation invariants for degree 1, 2, 3, 4 homogeneous polynomials and mixed. Degree $r$ vertex corresponds to order-$r$ tensor e.g. in polynomial describing shape. Operating on commutative field like $\mathbb{R}$ here, edges for given vertex are indistinguishable. Every edge corresponds to summation over corresponding index like in matrix product, and is rotation invariant thanks to $\sum_i O_{ai}O_{b i}=\delta_{a b}$ relation for rotation $O$ applied to both indexes of given edge. Invariants from disconnected graphs can be omitted as being products over invariants for its components. Fig. \ref{['aut']} proposes systematic generation of larger numbers of such invariants.
• Figure 4: Some possibilities for systematic generation of large numbers of rotation invariants from pnp, e.g. building $d\times d$ matrix $M$ (or $d^2 \times d^2$ matrix $C$) from order-3,4 tensors, then testing equality of $\textrm{Tr}(M^i)$ for $i=1,..,d$ (or $\textrm{Tr}(C^i)$ for $i=1,..,d^2$) for two tensors/polynomias/shapes, ensuring existence of orthogonal $d\times d$ (or $d^2 \times d^2$) between them. The dimension of $\textrm{SO}(d)$ rotations is $d(d-1)/2$, hence it should be sufficient to test this number of rotation invariants, however, mane of them are dependent like $\textrm{Tr}\left([p]^{i}\right)$ for $i=1,\ldots,d+1$ - the main difficulty is finding such number of independent.