The geometry of PLS shrinkages

TL;DR

This paper analyzes the geometry of shrinkages in Partial Least Squares ($PLS$) regressions. It derives an explicit, observation-driven formula for the shrinkage vector $\omega$ as a weighted average of a finite set of extreme shrinkage vectors $\omega_{(\tau)}$, with weights that are multilinear in $\psi = Y^2 \mathbf{1}$ and depend on the eigenstructure via $\pi_{\tau}$. The author then develops a geometric framework around the auxiliary variables $z = \mathbf{1} - \omega$ and $\alpha$, proving that shrinkages inhabit a convex-hull-like structure, while also describing the inverse mapping to the cone of feasible $\psi$. The work further shows that highly nonlinear shrinkage patterns can produce large expansions, leading to problematic Generalised Degrees of Freedom (GDoF) measures and contradicting prior conjectures that $\mathrm{GDoF} > n$ for PLS, supported by theoretical results and numerical examples. Altogether, the paper provides a rigorous foundation for understanding and quantifying the inner structure of $PLS$ estimators and motivates future distributional inference for shrinkage-based diagnostics.

Abstract

The geometrical structure of PLS shrinkages is here considered. Firstly, an explicit formula for the shrinkage vector is provided. In that expression, shrinkage factors are expressed a averages of a set of basic shrinkages that depend only on the data matrix. On the other hand, the weights of that average are multilinear functions of the observed responses. That representation allows to characterise the set of possible shrinkages and identify extreme situations where the PLS estimator has an highly nonlinear behaviour. In these situations, recently proposed measures for the degrees of freedom (DoF), that directly depend on the shrinkages, fail to provide reasonable values. It is also shown that the longstanding conjecture that the DoFs of PLS always exceeds the number PLS directions does not hold.

Figures (4)

• Figure 1: Geometry of the oblique projection $Q(\psi)$. The dotted and dashed lines represent, respectively, $\mathop{\mathrm{span}}\nolimits(\Lambda V)$ and $\mathbf{1} + \mathop{\mathrm{span}}\nolimits(\Lambda V)$. The thick segment corresponds to the vector $z$ and the other lines to $\mathop{\mathrm{span}}\nolimits(\Psi V)$ and $\mathbf{1} + \mathop{\mathrm{null}}\nolimits(\Psi V)$.
• Figure 2: Geometry of $z_{(\tau)}$ for $m=5,n=2$. Each line represents an affine space $\mathcal{Z}_i$.
• Figure 3: Geometry of $z_{(\tau)}$ for $m=6$, $n=3$ projected on the affine space $\mathcal{A}$. Each affine space $\mathcal{Z}_i$ is shown in a different color except for $\mathcal{Z}_1$ and $\mathcal{Z}_6$ which are not visible here. Lines correspond to the affine spaces $\mathcal{Z}_i \cap \mathcal{Z}_j$, $i \neq j$.
• Figure 4: CDF of the $\widehat{\mathop{\mathrm{GDoF}}\nolimits}$ and $\widehat{\mathop{\mathrm{GDoF}}\nolimits}_{DP}$ when $\lambda=(3.18,0.98,0.41,0.24,0.18)$, $\beta = (0.1,0.01,0.01,5,5)$, $\sigma=.02$ and $n=3$ and estimated by a MC experiment with $20\ 000$ replications.

Theorems & Definitions (13)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof