Beyond Coordinates: Meta-Equivariance in Statistical Inference

TL;DR

This work investigates whether coordinate choices influence the optimal decision when combining two asymptotically normal estimators. It shows that under strictly convex, differentiable objectives, the unique optimiser in the weight space transforms covariantly under any invertible affine reparameterisation, while the resulting optimal estimator remains invariant in the estimator space. The key results are Theorem 1 (estimator invariance under direct reparameterisation) and Theorem 2 (meta-equivariance: covariant transformation of the optimum under affine maps), validated both analytically and numerically. This establishes a coordinate-free notion of optimality in convex inference, connected to information geometry and convex geometry, and underscores the robustness of optimal decisions to how the problem is parameterised. The findings offer a geometric perspective on statistical decision-making, suggesting broader applicability to penalised estimation and model averaging where convexity and affine structure prevail.

Abstract

Optimal statistical decisions should transcend the language used to describe them. Yet, how do we guarantee that the choice of coordinates - the parameterisation of an optimisation problem - does not subtly dictate the solution? This paper reveals a fundamental geometric invariance principle. We first analyse the optimal combination of two asymptotically normal estimators under a strictly convex trace-AMSE risk. While methods for finding optimal weights are known, we prove that the resulting optimal estimator is invariant under direct affine reparameterisations of the weighting scheme. This exemplifies a broader principle we term meta-equivariance: the unique minimiser of any strictly convex, differentiable scalar objective over a matrix space transforms covariantly under any invertible affine reparameterisation of that space. Distinct from classical statistical equivariance tied to data symmetries, meta-equivariance arises from the immutable geometry of convex optimisation itself. It guarantees that optimality, in these settings, is not an artefact of representation but an intrinsic, coordinate-free truth.

Figures (4)

• Figure 1: Invariant estimator under affine reparameterisation. Black triangles mark base estimators $\hat{\theta}_1$ and $\hat{\theta}_2$, spanning the affine subspace of possible combinations. The red star shows the unique optimal estimator $\hat{\theta}^*_{\text{opt}}$, lying on this span. Dashed arrows depict the estimator maps $\mathcal{E}_A(W_{A,\text{opt}})$ and $\mathcal{E}_B(W_{B,\text{opt}})$ from the two parameterisations. Despite distinct weights, both maps yield the same point, confirming Theorem \ref{['thm:param_invariance']}: the estimator is invariant, even as its coordinates change.
• Figure 2: Conceptual risk geometry in parameter space. Contours represent level sets of the strictly convex risk $R(W)$. Markers denote optimal weights $W_{A,\text{opt}}$ and $W_{B,\text{opt}}$ under two affine parameterisations related by $T$. The affine map $T$ transforms the optimal weight covariantly ($W_{B,opt} = T(W_{A,opt})$), illustrating Theorem \ref{['thm:meta_equivariance']}. Strict convexity ensures a unique minimum point whose coordinates change predictably under $T$.
• Figure 3: Risk geometry projected onto a scalar subspace ($W=wI$). Curve A (blue) plots trace-AMSE $R(wI)$ vs $w$ (weight on $\hat{\theta}_2$). Curve B (orange, dashed) plots $R((1-w)I)$ vs $w$ (weight on $\hat{\theta}_1$). The mirrored shapes reveal meta-equivariance in 1D: symmetry $g(w) = f(1 - w)$ shows optimal weights shift ($w_B^* \approx 1 - w_A^*$) yet describe the same solution. Both minima trace back to the same invariant estimator.
• Figure 4: Commutative diagram illustrating estimator invariance. The affine map $T(W) = I - W$ transforms the weight parameterisation from $\mathcal{W}_A$ (weights on $\hat{\theta}_2$) to $\mathcal{W}_B$ (weights on $\hat{\theta}_1$). The mappings $\mathcal{E}_A(W_A) = (I-W_A)\hat{\theta}_1 + W_A\hat{\theta}_2$ and $\mathcal{E}_B(W_B) = W_B\hat{\theta}_1 + (I-W_B)\hat{\theta}_2$ map optimal weights to the estimator space $\mathbb{R}^K$. Meta-equivariance (Theorem \ref{['thm:meta_equivariance']}) ensures $W_{B,opt} = T(W_{A,opt})$. The diagram commutes ($\mathcal{E}_A = \mathcal{E}_B \circ T$ evaluated at optima), meaning both paths lead to the same invariant estimator $\hat{\theta}^*$, realising Theorem \ref{['thm:param_invariance']}. For a category-theoretic perspective, see fong2019.

Theorems & Definitions (12)

• Remark 2.1: Affine Structure and Coordinate Charts
• Theorem 2.2: Estimator Invariance Under Direct Reparameterisation
• proof
• Remark 2.3: Coordinate-Free Optimality
• Remark 2.4: The Geometric Engine: Strict Convexity
• Corollary 2.5
• Example 2.6: Illustrative
• Theorem 3.1: Meta-Equivariance under Affine Reparameterisation
• proof
• Remark 3.2: Geometric Interpretation