Trans-Glasso: A Transfer Learning Approach to Precision Matrix Estimation

TL;DR

This work proposes Trans-Glasso, a two-step transfer learning method for precision matrix estimation that derives the minimax optimal rate for differential network estimation, representing the first such guarantee in this area.

Abstract

Precision matrix estimation is essential in various fields, yet it is challenging when samples for the target study are limited. Transfer learning can enhance estimation accuracy by leveraging data from related source studies. We propose Trans-Glasso, a two-step transfer learning method for precision matrix estimation. First, we obtain initial estimators using a multi-task learning objective that captures shared and unique features across studies. Then, we refine these estimators through differential network estimation to adjust for structural differences between the target and source precision matrices. Under the assumption that most entries of the target precision matrix are shared with source matrices, we derive non-asymptotic error bounds and show that Trans-Glasso achieves minimax optimality under certain conditions. Extensive simulations demonstrate Trans Glasso's superior performance compared to baseline methods, particularly in small-sample settings. We further validate Trans-Glasso in applications to gene networks across brain tissues and protein networks for various cancer subtypes, showcasing its effectiveness in biological contexts. Additionally, we derive the minimax optimal rate for differential network estimation, representing the first such guarantee in this area.

Figures (7)

• Figure 1: Illustration of Assumption \ref{['assump:model-structure']}. The target precision matrix, $\Omega^{(0)}$, is shown alongside two source precision matrices, $\Omega^{(1)}$ and $\Omega^{(2)}$. Black crosses represent the shared entries across the matrices, while colored shapes indicate individual, unique entries.
• Figure 2: Simulation results for Model I. The default setting is $n_0=300$, $n_{\text{source}}=1000$ and $h=40$. In the first experiment, we increase $n_0$ while fixing $n_{\text{source}}$ and $h$. In the second experiment, we increase $n_{\text{source}}$ while fixing $n_0$ and $h$. In the third experiment, we increase both $n_0$ and $n_{\text{source}}$ while increasing $h$. More specifically, we let $n_{\text{source}}=3 n_0$, and $n_0=70$ when $h=10$, $n_0=150$ when $h=20$, $n_0=300$ when $h=30$, $n_0=600$ when $h=40$ and $n_0=1200$ when $h=50$. In the fourth experiment, we fix both $n_0$ and $n_{\text{source}}$ while increasing $h$. Each dot represents the empirical mean across $30$ repetitions and the vertical bar represents $\text{Mean} \pm \frac{2}{\sqrt{30}} \times \text{Standard Error}$.
• Figure 3: Simulation results for Model II. The default setting is $n_0=750$, $n_{\text{source}}=2000$ and $h=40$. In the first experiment, we increase $n_0$ while fixing $n_{\text{source}}$ and $h$. In the second experiment, we increase $n_{\text{source}}$ while fixing $n_0$ and $h$. In the third experiment, we increase both $n_0$ and $n_{\text{source}}$ while increasing $h$. More specifically, we let $n_{\text{source}}=3 n_0$, and $n_0=100$ when $h=20$, $n_0=200$ when $h=30$, $n_0=300$ when $h=40$, $n_0=500$ when $h=50$, $n_0=800$ when $h=60$, $n_0=1000$ when $h=70$ and $n_0=1200$ when $h=80$. In the fourth experiment, we fix both $n_0$ and $n_{\text{source}}$ while increasing $h$. Each dot represents the empirical mean across $30$ repetitions and the vertical bar represents $\text{Mean} \pm \frac{2}{\sqrt{30}} \times \text{Standard Error}$.
• Figure 4: Simulation results for Model III. The default setting is $n_0=150$, $n_{\text{source}}=1000$ and $h=40$. In the first experiment, we increase $n_0$ while fixing $n_{\text{source}}$ and $h$. In the second experiment, we increase $n_{\text{source}}$ while fixing $n_0$ and $h$. In the third experiment, we increase both $n_0$ and $n_{\text{source}}$ while increasing $h$. More specifically, we let $n_{\text{source}}=4 n_0$, and $n_0=15$ when $h=20$, $n_0=30$ when $h=30$, $n_0=80$ when $h=40$, $n_0=300$ when $h=50$, and $n_0=1000$ when $h=60$. In the fourth experiment, we fix both $n_0$ and $n_{\text{source}}$ while increasing $h$. Each dot represents the empirical mean across $30$ repetitions and the vertical bar represents $\text{Mean} \pm \frac{2}{\sqrt{30}} \times \text{Standard Error}$.
• Figure 5: Simulation results when the informative set $\mathcal{A}$ is unknown. We set $n_0=300$ and $n_{\text{source}}=1000$ for Model I; $n_0=750$ and $n_{\text{source}}=2000$ for Model II; and $n_0=300$ and $n_{\text{source}}=1000$ for Model III. Each dot represents the empirical mean across $30$ repetitions and the vertical bar represents $\text{Mean} \pm \frac{2}{\sqrt{30}} \times \text{Standard Error}$.

Theorems & Definitions (39)

• Theorem 1
• Theorem 2
• Theorem 3
• Corollary 1
• Theorem 4
• Theorem 5
• Theorem 6
• Lemma 1
• proof
• Lemma 2