Multifidelity Covariance Estimation via Regression on the Manifold of Symmetric Positive Definite Matrices

TL;DR

It is shown that the Riemannian regression framework encompasses existing multifidelity covariance estimators constructed from control variates, and preservation of positive definiteness ensures that the estimator is compatible with downstream tasks, such as data assimilation and metric learning, in which this property is essential.

Abstract

We introduce a multifidelity estimator of covariance matrices formulated as the solution to a regression problem on the manifold of symmetric positive definite matrices. The estimator is positive definite by construction, and the Mahalanobis distance minimized to obtain it possesses properties enabling practical computation. We show that our manifold regression multifidelity (MRMF) covariance estimator is a maximum likelihood estimator under a certain error model on manifold tangent space. More broadly, we show that our Riemannian regression framework encompasses existing multifidelity covariance estimators constructed from control variates. We demonstrate via numerical examples that the MRMF estimator can provide significant decreases, up to one order of magnitude, in squared estimation error relative to both single-fidelity and other multifidelity covariance estimators. Furthermore, preservation of positive definiteness ensures that our estimator is compatible with downstream tasks, such as data assimilation and metric learning, in which this property is essential.

Figures (8)

• Figure 1: Intrinsic MSE of ${\hat{\Sigma}_{\text{hi}}}$ (red) and mean Mahalanobis distance at ${\hat{\Sigma}_{\text{hi}}}$ (teal) as a function of regularization parameter $\lambda$ in the fixed-${\Sigma_{\rm lo}}$ setting. We vary the dimension $d \in \{3, 4, 5\}$ within a class of simple example problems. The $\lambda$ associated with the minimum of the MSE curves corresponds closely to that associated with mean Mahalanobis distance equal to $\frac{d(d+1)}{2}$, plotted with dashed black lines.
• Figure 1: Simple Gaussian example: Regularization parameters selected by matching mean minimum Mahalanobis distance over 32 pilot trials to $\frac{d(d+1)}{2}$ (left), resulting mean minimum Mahalanobis distance over 3000 trials using the selected regularization parameters (middle), and fraction of EMF estimators which were indefinite over 3000 repeated trials (right). All budgets except $B = 196$ resulted in at least one indefinite EMF estimator.
• Figure 2: Simple Gaussian example: Median squared error in the Frobenius norm (left) and intrinsic metric (right).
• Figure 3: Simple Gaussian example: Frobenius squared error histograms of ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ compared to ${\hat{\Sigma}_{\text{hi}}}^{\rm HF}$ (top), ${\hat{\Sigma}_{\text{hi}}}^{\rm LF}$ (middle), and ${\hat{\Sigma}_{\text{hi}}}^{\rm LEMF}$ (bottom). ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ attains significantly lower error than ${\hat{\Sigma}_{\text{hi}}}^{\rm HF}$ at all budgets, intuitively because it obtains more information, via recourse to correlated low-fidelity samples, at the same cost. For small budgets ${\hat{\Sigma}_{\text{hi}}}^{\rm LF}$ has lower squared error than ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ because its variability is small due to the large number of samples comprising it, but as the budget increases its bias becomes apparent and ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ yields estimates with lower error.
• Figure 4: Simple Gaussian example: Intrinsic squared error distributions of ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ as compared to ${\hat{\Sigma}_{\text{hi}}}^{\rm HF}$ (top), ${\hat{\Sigma}_{\text{hi}}}^{\rm LF}$ (middle), and ${\hat{\Sigma}_{\text{hi}}}^{\rm LEMF}$ (bottom). The advantages of ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ relative to ${\hat{\Sigma}_{\text{hi}}}^{\rm HF}$ and ${\hat{\Sigma}_{\text{hi}}}^{\rm LF}$ are more pronounced in the intrinsic metric, which compares matrices as operators by examining their generalized eigenvalues, than in the Frobenius metric, which compares matrices as vectors. The intrinsic metric also reveals the poor performance of ${\hat{\Sigma}_{\text{hi}}}^{\rm LEMF}$ at low budgets, though at higher budgets the performances of ${\hat{\Sigma}_{\text{hi}}}^{\rm MRMF}$ and ${\hat{\Sigma}_{\text{hi}}}^{\rm LEMF}$ are comparable.

Theorems & Definitions (15)

• Definition 3.1: Manifold Regression Multifidelity (MRMF) Covariance Estimator
• Proposition 4.1
• Proposition 4.2
• Proposition 4.3
• Proposition 4.4
• Proposition 4.5
• Theorem 4.6
• Lemma A.1
• Proof 1
• Lemma A.2