Estimating the normal-inverse-Wishart distribution

TL;DR

The paper addresses the problem of converting between mean and natural parameters in the normal-inverse-Wishart (NIW) exponential-family representation, and of performing maximum likelihood estimation of the natural parameters from observed sufficient statistics. It presents the forward mapping, where $E_p[s(\mu,\Sigma)]=\nabla A(\eta)$, yielding explicit expressions for the mean-parameter witnesses $m_1, m_2, m_3, m_4$ in terms of the natural parameters $\eta=(\text{vec}(\Psi+\lambda\mu_0\mu_0^\top), \lambda\mu_0, \lambda, \nu)$. The reverse mapping is tackled by showing $\nu$ must be found via one-dimensional optimization of a strictly increasing, strictly concave function $f(\nu)$, ensuring a unique root and enabling a convergent Newton-Raphson search; once $\nu$ is obtained, $\mu_0$, $\lambda$, and $\Psi$ follow from the remaining equations. The contribution provides a robust, convergent procedure for parameter conversion in NIW models, facilitating efficient use of NIW priors in expectation propagation and related Bayesian inference tasks.

Abstract

The normal-inverse-Wishart (NIW) distribution is commonly used as a prior distribution for the mean and covariance parameters of a multivariate normal distribution. The family of NIW distributions is also a minimal exponential family. In this short note we describe a convergent procedure for converting from mean parameters to natural parameters in the NIW family, or -- equivalently -- for performing maximum likelihood estimation of the natural parameters given observed sufficient statistics. This is needed, for example, when using a NIW base family in expectation propagation.