Precision Neural Networks: Joint Graph And Relational Learning

TL;DR

This work addresses the limitations of covariance-based graph learning by proposing Precision Neural Networks (PNNs) that operate on the inverse covariance $\mathbf{\Theta}$ to leverage sparsity and conditional independence. It introduces a task-aware, alternating-optimization framework to jointly learn the PNN weights and the sparse precision matrix, with a relaxed formulation that decouples updates via an auxiliary $\tilde{\mathbf{\Theta}}$ and a graphical-Lasso–like objective. A theoretical bound shows the precision estimate converges to the true matrix at rate $O(T^{-1/2})$, improved by sparsity and alignment with downstream tasks. Empirical results on synthetic data and neuroimaging datasets (age prediction from cortical thickness) demonstrate that joint optimization yields superior downstream performance and sparser, more informative precision estimates than two-step or sample-based baselines. Overall, the method offers a principled, task-tailored approach to structure learning in high-dimensional settings with practical implications for neuroscience and related fields.

Abstract

CoVariance Neural Networks (VNNs) perform convolutions on the graph determined by the covariance matrix of the data, which enables expressive and stable covariance-based learning. However, covariance matrices are typically dense, fail to encode conditional independence, and are often precomputed in a task-agnostic way, which may hinder performance. To overcome these limitations, we study Precision Neural Networks (PNNs), i.e., VNNs on the precision matrix - the inverse covariance. The precision matrix naturally encodes statistical independence, often exhibits sparsity, and preserves the covariance spectral structure. To make precision estimation task-aware, we formulate an optimization problem that jointly learns the network parameters and the precision matrix, and solve it via alternating optimization, by sequentially updating the network weights and the precision estimate. We theoretically bound the distance between the estimated and true precision matrices at each iteration, and demonstrate the effectiveness of joint estimation compared to two-step approaches on synthetic and real-world data.

Figures (2)

• Figure 1: Data processing via covariance information. Traditional pipelines use PCA for preprocessing followed by an ML model for the downstream task. VNNs and PNNs merge these steps by learning end-to-end on the covariance or the precision graph. Joint PNNs unify the estimation of the precision matrix and the network weights.
• Figure 2: Regression error (MAE and MSE), precision matrix reconstruction quality in terms of L1 norm and sparsity of estimated precision $\mathbf{\Theta}$ on the synthetic datasets for varying sparsity of the true precision $\mathbf{\Theta}_0$.