On Procrustes Contamination in Machine Learning Applications of Geometric Morphometrics

TL;DR

The paper addresses data leakage in machine learning applications of geometric morphometrics caused by Generalised Procrustes Analysis (GPA) aligning all specimens before train–test split. It formalizes Procrustes contamination, validates it with controlled 2D and 3D simulations across varying sample sizes, landmark densities, and allometric patterns, and introduces a test-set realignment workflow to eliminate cross-sample dependency. A key finding is a diagonal relationship between the number of landmarks and samples that characterizes RMSE scaling in Procrustes space, with slopes near 1/3 in 2D and 1/4 in 3D, alongside evidence that preserving spatial autocorrelation among landmarks improves predictive performance. The work provides practical preprocessing guidelines, demonstrates the fundamental statistical constraints of Procrustes shape space, and highlights potential graph-based approaches to better exploit landmark covariation in morphometric ML analyses.

Abstract

Geometric morphometrics (GMM) is widely used to quantify shape variation, more recently serving as input for machine learning (ML) analyses. Standard practice aligns all specimens via Generalized Procrustes Analysis (GPA) prior to splitting data into training and test sets, potentially introducing statistical dependence and contaminating downstream predictive models. Here, the effects of GPA-induced contamination are formally characterised using controlled 2D and 3D simulations across varying sample sizes, landmark densities, and allometric patterns. A novel realignment procedure is proposed, whereby test specimens are aligned to the training set prior to model fitting, eliminating cross-sample dependency. Simulations reveal a robust "diagonal" in sample-size vs. landmark-space, reflecting the scaling of RMSE under isotropic variation, with slopes analytically derived from the degrees of freedom in Procrustes tangent space. The importance of spatial autocorrelation among landmarks is further demonstrated using linear and convolutional regression models, highlighting performance degradation when landmark relationships are ignored. This work establishes the need for careful preprocessing in ML applications of GMM, provides practical guidelines for realignment, and clarifies fundamental statistical constraints inherent to Procrustes shape space.

Figures (5)

• Figure 1: Illustrative examples of underfitting (left), near-optimal fitting (center), and overfitting (right) in regression on synthetic data. Top row: linear relationship $y = 2x - 1 + \mathcal{N}(0, 0.4^2)$. Middle row: moderately nonlinear relationship $y = \sin(2\pi x) + 0.2x + \mathcal{N}(0, 0.4^2)$. Bottom row: highly nonlinear relationship $y = \sin(6\pi x) + \tfrac{1}{2}\sin(14\pi x) + \mathcal{N}(0, [0.25(1+x)]^2)$, exhibiting heteroscedastic noise.
• Figure 2: Bootstrapped ($\times$1000) Procrustes distances between Procrustes-aligned coordinates of specimens common to both datasets when GPA is performed on the full sample versus a reduced sample obtained by removing a single individual. Procrustes distances increase as sample size decreases, demonstrating that Procrustes-aligned coordinates are sensitive to sample composition.
• Figure 3: $RMSE$ and $\Delta RMSE$ values comparing the two pipelines; the contaminated GPA approach where GPA is performed prior to the train-test split, and the clean approach where GPA is performed on the training data, and test data is later projected onto the already superimposed data.
• Figure 4: RMSE values comparing sample size with the number of landmarks under different experimental constraints in both 2D and 3D
• Figure 5: A boxplot comparing the RMSE values of 300 training simulations of basic linear models on landmark data and a model that is spatially aware of landmark coordinates.

Theorems & Definitions (1)

• Proposition 7.1