Asymptotic spectrum of weighted sample covariance: another proof of spectrum convergence

TL;DR

This paper provides a concise, self-contained proof of the almost-sure convergence of the spectrum of the weighted sample covariance in the high-dimensional regime, under a set of assumptions that are broader yet robust. It extends the Marčenko–Pastur framework to weighted covariances by deriving a deterministic limit F whose Stieltjes transform m is characterized by a pair of coupled equations involving the limit laws H and D of the population spectrum and the weights. A key novelty is the introduction of the auxiliary function \tilde{m}(z) that deforms the limit through a fixed-point equation, enabling a clearer, pedagogical derivation compared to prior work. The results are illustrated with various weight distributions (uniform, mixtures, EWMA) and are shown to capture phenomena like spectral separation and the impact of heavy tails on finite-sample behavior, highlighting practical implications for high-dimensional covariance estimation under weighted schemes.

Abstract

We propose another proof of the high dimensional spectrum convergence of the weighted sample covariance, more concise and self-sufficient but with stronger, but reasonable assumptions. We explain and illustrates this theorem for different weight distributions and show how the spectrum behaves in finite samples with heavy tails. The general purpose is to provide a detailed introduction to the high dimensional spectrum of weighted sample covariance.

Figures (2)

• Figure 1: Different approximations regarding parameter $\alpha$ of the asymptotic density of weighted sample eigenvalues for $c=0.1$, true covariance spectrum distribution $H = 1_{[1,\infty[}$ and weight distribution $D_\alpha = \frac{1}{2} 1_{[1-\alpha,\infty[} + \frac{1}{2} 1_{[1+\alpha,\infty[}$. $\alpha = 0$ corresponds to the classic Marcenko-Pastur density for $c=0.1$.
• Figure 2: Different approximations regarding parameter $\alpha$ of the asymptotic density of weighted sample eigenvalues for $c=0.1$, true covariance spectrum distribution $H = 1_{[1,\infty[}$ and weight distribution $D_\alpha: x \in [\beta e^{-\alpha}, \beta] \mapsto 1 + \frac{1}{\alpha}\log\left(\frac{x}{\beta}\right) \text{ with } \beta = \frac{\alpha}{1- e^{-\alpha}}$, defined in Definition \ref{['ewma']}. $\alpha = 0$ corresponds to the classic Marcenko-Pastur density for $c=0.1$.

Theorems & Definitions (6)

• Example 1: Standard weighting
• Example 2: Exponentially weighted scheme
• Theorem 1: Cauchy-Stieltjes convergence, Theorem 5.8.3 Fleermann2022
• Theorem 2: Asymptotic spectrum
• Remark 1: Role of $\tilde{m}$
• Remark 2: Role of $\tilde{m}$ when $D = 1_{[1,+\infty[}$] When $D = 1_{[1,+\infty[}$, we have that for all $z \in \mathbb{C}_+$, $\tilde{m}(z) = 1 - c\left(1+zm(z)\right) = -z \underline{m}(z)$. The conclusion of this remark is that $\tilde{m}$ in the setting of weighted sample covariance has a similar role than $\underline{m}$ has in the uniformly weighted setting. We show the behavior of the asymptotic density when we vary the weight distribution. The reference distribution of true eigenvalues is $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$, introduced by Bai and Silverstein Bai1998 and used by Ledoit and Péché Ledoit2009. This distribution helps visualizing the phenomenon of spectral separation, which is deeply studied in Couillet2015: areas of exclusion between the true eigenvalues where the asymptotic density is null. Those areas vanish when $c$ is increasing, as was shown in the classic setting of Marcenko-Pastur theory, but we can see here that the same phenomenon appears when the weight distribution is smoothly spreading. For that matter, we are considering different weight distribution parametrized by $\alpha \in \mathbb{R}_+$ where $\alpha$ controls how much we are spreading the weights. We define the exponentially-weighted distribution, that corresponds to the distribution of weights in an Exponentially-Weighted Moving Average - EWMA -, used in time series analysis. We fix $\alpha \in \mathbb{R}_+$. We define this law such as its cdf $D$ follows: $D(\beta e^{-\alpha t}) = 1 - t$ for $t \in [0,1]$. Moreover, we impose that $\int \delta dD(\delta) = 1$. We finally have: $\forall x \in [\beta e^{-\alpha}, \beta], D(x) = 1 + \frac{1}{\alpha} \log\left(\frac{x}{\beta}\right)$, with $\beta = \frac{\alpha}{1 - e^{-\alpha}}$. We consider three weight distributions: A uniform distribution with $[1 - \alpha/2, 1 + \alpha/2]$ for $\alpha \in [0,2]$. The experiment is shown in figure \ref{['fig:sep_unif']}. There are 2 spectral separation for $\alpha = 0$ and $\alpha=1$. For $\alpha = 2$ there is only one left.A mixture of 2 diracs, with $D = \frac{1}{2} 1_{[1-\alpha,\infty[} + \frac{1}{2} 1_{[1+\alpha,\infty[}$ for $\alpha \in [0,1]$. The experiment is shown in figure \ref{['fig:sep_2diracs']}. There are 2 spectral separation for $\alpha = 0$. For $\alpha = 0.7$ and $\alpha = 1$ there is no more spectral separation.An exponentially-weighted distribution. The experiment is shown in figure \ref{['fig:sep_exp']}. There are 2 spectral separation for $\alpha = 0$, only 1 for $\alpha=2$ and no more for $\alpha = 5$. Approximations of the asymptotic density of weighted sample eigenvalues for $c=0.25$,with $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and uniform weight distribution of parameter $\alpha$. Approximations of the asymptotic density of weighted sample eigenvalues for $c=0.25$, with $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and weight distribution from a mixture of 2 diracs of parameter $\alpha$. Approximations of the asymptotic density of weighted sample eigenvalues for $c=0.25$, with $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and exponentially-weighted distribution of parameter $\alpha$. Silverstein and Choi Silverstein1995c proved, in the case of the non-weighted sample covariance, that when $H$ is a finite sum of $K$ diracs, then there are at most $K-1$ spectral gaps in the asymptotic spectrum $F$. In the case of the weighted covariance spectrum, this result is not true anymore, as gaps in $D$ can lead to new spectral gaps in $F$, sa explained theoretically in Couillet2015. To illustrate this new phenomenon, we consider: $c = 0.05$ and $c=0.01$,$H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$,$D = \left(1 - \frac{1}{2\alpha} \right) \times 1_{[\frac{\alpha}{2\alpha - 1},\infty[} + \frac{1}{2\alpha} 1_{[\alpha,\infty[}$ for $\alpha \in \{1, 50 \}$ with $c = 0.05$ and $\alpha \in \{1, 200 \}$ with $c = 0.01$. In the standard non-weighted sample covariance case, we expect to find two spectral gaps roughly when $c<0.34$, one spectral gap for $c \in [0.34,0.40]$ and no spectral gaps for $c>0.40$. These values result from the theoretical study of Silverstein and Choi Silverstein1995c. In our setting, we indeed have at most two spectral gaps with $\alpha = 1$, which coincides with the non-weighted situation. However, for $\alpha = 50$ and $\alpha = 200$, a third spectral gap appears at smaller values of $c$, respectively numerically for $c\leq 0.05$ and $c\leq 0.01$. Results are shown in figure \ref{['fig:sep_3']}. Approximations of the asymptotic density of weighted sample eigenvalues for $c=0.05$ and $c=0.01$, with $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and weight distribution from a mixture of 2 diracs of parameter $\alpha$ for spectral gaps. This last experimental part aims at empirically showing how the difference in finite sample to the asymptotic density evolves depending on the heaviness of tails. To visualize this point, we are plotting the histogram of the empirical spectrum of the weighted sample covariance along with the theoretical asymptotic density. We draw the $(X_{ij})$ from a normalized t-distribution with varying degree of freedom $\nu \in \{2.5, 3.5, 4\}$. As previously, we chose $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and an exponentially-weighted distribution of parameter $\alpha = 1$ for the weights. The concentration ratio is set at $c=0.25$ and the dimension at $n = 3000$. The convergence is meant to be almost sure, so we did only one draw of spectrum per plot. The results are shown in figure \ref{['fig:hist_exp']}. The conclusion of this experiment is that heavy tails tend to create rare but high eigenvalues, making the convergence slower as the tail grows. We recall that the convergence of the distributions is shown under weak convergence. So, the presence of very high eigenvalues far from the support of $F$ is not a contradiction as long as their frequence of appearance converges to zero, which experimentally is the case. This type of behavior is clearly visible for $\nu = 2.5$ in Figure \ref{['fig:hist_exp']} where eigenvalues higher than $40$ are sampled, but in very low proportion. Histogram from a t-distribution with $\nu = 4, \nu = 3$ and $\nu=2.5$ from top to bottom, and asymptotic theoretical density, for $n = 3000$, $c=0.25$, $H = 0.2 \times 1_{[1,\infty[} + 0.4 \times 1_{[3,\infty[} + 0.4 \times 1_{[10,\infty[}$ and exponentially-weighted distribution of parameter $\alpha = 1$. In this work, we provide a detailed introduction to the high dimensional spectrum of weighted sample covariance. We expose the setting, the convergence result in high dimension, along with a proof arguably simpler and more concise than the more optimal proof of Zhang2007, and we illustrate some basic phenomenons to give the reader the correct intuition on what is happening in high dimensions when weights get involved in the estimation. We would like to thank Alexandre Miot and Gabriel Turinici for their insights and advices all along this work. In this first part of the proof, we use a similar approach as Silverstein1995b, aiming at truncating and centering $Z$, $T$ and $W$ while conserving the same asymptotic spectrum for $F_n$. Let $\hat{Z}_{ij} = Z_{ij} \mathrm{1}_{|Z_{ij}| < \sqrt{N}}$ and $\hat{\underline{B}}_n = \frac{1}{N} W^{1/2} \hat{Z}^* T \hat{Z} W^{1/2}$. We use Lemmas 2.5.a and 2.1.d Silverstein1995b, and with $\lVert \cdot \rVert$ denoting the $\infty$-norm on bounded real functions, we have: $\lVert F^{\underline{B}_n} - F^{\hat{\underline{B}}_n} \rVert\leq \frac{2}{N} \text{rank}\left(W^{1/2}Z - W^{1/2} \hat{Z} \right)\leq \frac{2}{N} \text{rank}\left(Z - \hat{Z} \right)\lVert F^{\underline{B}_n} - F^{\hat{\underline{B}}_n} \rVert\leq \frac{2}{N} \underbrace{\left|\left \{ i \in \llbracket 1,n \rrbracket, j \in \llbracket 1,N \rrbracket | Z_{ij} - \hat{Z}_{ij} \neq 0 \right \} \right|}_{\xi_n}$ At this point, we follow the idea of the proof p.58-59 of Yin1986. We have that: $\xi_n = \sum_{i,j} Y_{i,j} \text{ with } Y_{ij} = \mathrm{1}_{|Z_{ij}| \geq \sqrt{N}}.$ Remark that the $Y_{ij}$ are i.i.d Bernoulli $\mathcal{B}(\eta)$ with $\eta = \mathbb{P}\left(|Z_{11}| \geq \sqrt{N} \right)$. As $\mathbb{E}[|Z_{11}|^2] = 1$, $\eta = o\left(\frac{1}{n}\right)$. Let $\epsilon > 0$. For $n$ large enough, we have: $\mathbb{P}\left(\frac{2}{N} \xi_n \geq \epsilon \right)= \mathbb{P}\left(\frac{1}{nN} \sum_{ij} Y_{ij} \geq \frac{\epsilon}{2n} \right) \leq \mathbb{P}\left(\frac{1}{nN} \sum_{ij} Y_{ij}\geq \eta + \frac{\epsilon}{4n} \right).$ As $\mathbb{E}[Y_{ij}] = \eta$, we have by Chernoff-Hoeffding's theorem, for $n$ large enough: $\mathbb{P}\left(\frac{2}{N} \xi_n \geq \epsilon \right)\leq \exp\left[nN\left(\left(\eta + \frac{\epsilon}{4n} \right) \ln{\frac{\eta}{\eta + \frac{\epsilon}{4n}}} + \left(1 - \eta - \frac{\epsilon}{4n} \right) \ln{\frac{1-\eta}{1-\eta - \frac{\epsilon}{4n}}} \right) \right]\leq \exp\left[nN\left(- \frac{\left(\frac{\epsilon}{4n} \right)^2}{2\left(\eta + \frac{\epsilon}{4n} \right)} - \frac{\left(\frac{\epsilon}{4n} \right)^3}{3\left(\eta + \frac{\epsilon}{4n} \right)^2} - \frac{ \left(\frac{\epsilon}{4n} \right)^2}{ 2\left(1 - \eta - \frac{\epsilon}{4n} \right)} + \frac{ \left(\frac{\epsilon}{4n} \right)^3}{3 \left(1 - \eta - \frac{\epsilon}{4n} \right)^2} \right) \right]= \exp\left[- \frac{\epsilon^2N}{32n}\times \frac{1}{\left(\eta + \frac{\epsilon}{4n} \right)\left(1 - \eta - \frac{\epsilon}{4n} \right)} - \frac{\epsilon^3N}{3 \times 4^3n^2}\times \frac{1 - 2\left(\eta + \frac{\epsilon}{4n}\right)}{\left(\eta + \frac{\epsilon}{4n} \right)^2\left(1 - \eta - \frac{\epsilon}{4n} \right)^2}\right]\mathbb{P}\left(\frac{2}{N} \xi_n \geq \epsilon \right)\leq \exp\left[- \epsilon N \times \left(\frac{1}{16} + \frac{2}{3 \times 4^3} \right) \right].$ Consequently, $\lVert F^{\underline{B}_n} - F^{\hat{\underline{B}}_n} \rVert \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely.}$ Let $\tilde{Z}_{ij} = \hat{Z}_{ij} - \mathbb{E}\left[\hat{Z}_{ij} \right]$ and $\tilde{\underline{B}}_n = \frac{1}{N} W^{1/2} \tilde{Z}^* T \hat{Z} W^{1/2}$. The $\hat{Z}_{ij}$ are i.i.d, so $\forall i,j, \mathbb{E}\left[\hat{Z}_{ij} \right] = \mathbb{E}\left[\hat{Z}_{11} \right]$. Consequently, $\text{rank}\left(W^{1/2} \tilde{Z} - W^{1/2} \hat{Z} \right) = \text{rank}\left(W^{1/2} \mathbb{E}[\hat{Z}] \right) \leq 1.$ Using Lemma 2.5.a Silverstein1995b, we have: $\lVert F^{\tilde{\underline{B}}_n} - F^{\hat{\underline{B}}_n} \rVert \leq \frac{2}{N} \text{rank}\left(W^{1/2} \tilde{Z} - W^{1/2} \hat{Z} \right) \leq \frac{2}{N} \underset{n \rightarrow \infty}{\longrightarrow} 0.$ We denote $U$ a matrix of eigenvector of $T$ with associated eigenvalues $(\tau_1,...,\tau_n)$, so that $T = U \mathop{\mathrm{Diag}}\limits\left((\tau_i)\right) U^*$. For $\alpha > 0$, we define: $T_\alpha = U \mathop{\mathrm{Diag}}\limits\left(\left(\tau_i \mathrm{1}_{|\tau_i|\leq \alpha}\right)\right) U^*.$ If $\alpha$ and $-\alpha$ are continuity points of $H$, using Lemma 2.5.b Silverstein1995b, we have that for any $n \times N$ Hermitian matrix $Q$: $\lVert F^{Q^* T Q} - F^{Q^* T_\alpha Q} \rVert \leq \frac{1}{N} \text{rank}(T - T_\alpha) = \frac{1}{N} \sum_{i=1}^n \mathrm{1}_{|\tau_1|>\alpha}.$ Using Assumptions (b) and (c), we have then: $\frac{1}{N} \text{rank}(T - T_\alpha) \underset{n \rightarrow \infty}{\longrightarrow} cH([-\alpha,\alpha]^c) \text{ almost surely.}$ Consequently, $\alpha := \alpha_n \underset{n \rightarrow \infty}{\longrightarrow} +\infty \implies \lVert F^{Q^* T Q} - F^{Q^* T_{\alpha} Q} \rVert \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely.}$ Similarly, for $\beta > 0$, we define: $W_\beta = \mathop{\mathrm{Diag}}\limits\left(\left(W_{ii} \mathrm{1}_{|W_{ii}|\leq \beta}\right)\right).$ Using Lemma 2.5.b Silverstein1995b and Assumptions (b) and (c), we have also for any $n \times N$ Hermitian matrix $Q$: $\lVert F^{W^{1/2} Q^* T_\alpha Q W^{1/2}} - F^{W_\beta^{1/2} Q^* T_\alpha Q W_\beta^{1/2}} \rVert \leq \frac{1}{n} \text{rank}(W - W_\beta) \underset{n \rightarrow \infty}{\longrightarrow} \frac{1}{c}D([-\beta,\beta]^c) \text{ almost surely.}$ Consequently, $\beta := \beta_n \underset{n \rightarrow \infty}{\longrightarrow} +\infty \implies \lVert F^{W^{1/2} Q^* T_\alpha Q W^{1/2}} - F^{W_\beta^{1/2} Q^* T_\alpha Q W_\beta^{1/2}} \rVert \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely.}$ In the following, we choose $\alpha_n \uparrow +\infty$ and $\beta_n \uparrow +\infty$ so that: $(\alpha_n\beta_n)^4\left( \mathbb{E}\left[|Z_{11}|^2 \mathrm{1}_{|Z_{11}| \geq \ln N} \right] + \frac{1}{N}\right) \underset{n \rightarrow \infty}{\longrightarrow} 0,$$\sum_{n=1}^\infty \frac{(\beta_n\alpha_n)^8}{N^2} \left( \mathbb{E}\left[|Z_{11}|^4 \mathrm{1}_{|Z_{11}| \geq \sqrt{N}} \right] +1 \right) < +\infty.$ We denote $\tilde{\underline{B}}_{\alpha,\beta} = \frac{1}{N}W_\beta^{1/2} \tilde{Z}^* T_\alpha \tilde{Z} W_\beta^{1/2}$. With $\alpha, \beta$ chosen following the rules above, we have: $\lVert F^{\tilde{\underline{B}}_{\alpha,\beta}} - F^{\tilde{\underline{B}}_{n}} \rVert \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely.}$ Let $\bar{Z}_{ij} = \tilde{Z}_{ij} \mathrm{1}_{|Z_{ij}| < \ln N} - \mathbb{E}\left[\tilde{Z}_{ij} \mathrm{1}_{|Z_{ij}| < \ln N} \right]$ and $\bar{\bar{Z}}_{ij} = \tilde{Z}_{ij} - \bar{Z}_{ij}$. Remark that $\mathbb{E}[\bar{\bar{Z}}] = \mathbb{E}[\bar{Z}] = 0$. Moreover, since $|\tilde{Z}_{ij}| \leq \ln N + \mathbb{E}[|Z_{11}|]$, we have for $n$ sufficiently large and some $a > 2$: $\frac{|\bar{Z}_{ij}|}{\sqrt{\mathbb{E}[|\bar{Z}_{11}|^2]}} \leq \frac{\ln N + \mathbb{E}[|Z_{11}|]}{\sqrt{\mathbb{E}[|\bar{Z}_{11}|^2]}} \leq a \ln n := \log n.$ We denote $\log n$ the logarithm of $n$ in base $\exp(1/a)$. In this part, we use a metric $D$ on $\mathcal{M}(\mathbb{R})$, the set of all subprobability distribution functions on $\mathbb{R}$, defined in Silverstein1995b p.191. Let $\{f_i \}$ be an enumeration of all continuous functions that take a constant $1/m$ value ($m$ a positive integer) on $[a,b]$, where $a, b$ are rational, $0$ on $]-\infty, a-1/m] \cup [b+1/m, +\infty[$, and linear on each $[a-1/m,a]$, $[b,b+1/m]$. For $F_1, F_2 \in \mathcal{M}(\mathbb{R})$, we define: $D(F_1,F_2) := \sum_{i=1}^\infty \left|\int f_i dF_1 - \int f_idF_2 \right|2^{-i}.$ $D$ is a metric on $\mathcal{M}(\mathbb{R})$ inducing the topology of vague convergence. We denote $\bar{\underline{B}}_{\alpha,\beta} = \frac{1}{N} W_\beta^{1/2} \bar{Z}^* T_\alpha \bar{Z} W_\beta^{1/2}$. Using Lemma 2.1.c and Equation (2.4) Silverstein1995b, we have: $D^2\left(F^{\tilde{\underline{B}}_{\alpha,\beta}}, F^{\bar{\underline{B}}_{\alpha,\beta}} \right)\leq\frac{1}{N} \text{Tr} \left[\left( W_\beta^{1/2} \tilde{Z}^* T_\alpha \tilde{Z} W_\beta^{1/2} - W_\beta^{1/2} \bar{Z}^* T_\alpha \bar{Z} W_\beta^{1/2}\right)^2 \right]=\frac{1}{N} \text{Tr} \left[\left( W_\beta^{1/2} (\bar{\bar{Z}} + \bar{Z})^* T_\alpha (\bar{\bar{Z}} + \bar{Z}) W_\beta^{1/2} - W_\beta^{1/2} \bar{Z}^* T_\alpha \bar{Z} W_\beta^{1/2}\right)^2 \right]D^2\left(F^{\tilde{\underline{B}}_{\alpha,\beta}}, F^{\bar{\underline{B}}_{\alpha,\beta}} \right)=\frac{1}{N} ( \text{Tr} \left[\left( W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2}\right)^2\right] + \text{Tr}\left[\left(W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{Z} W_\beta^{1/2} + W_\beta^{1/2} \bar{Z} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2} \right)^2 \right]+ 2\text{Tr}\left[\left(W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2} \right)\left( W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{Z} W_\beta^{1/2} + W_\beta^{1/2} \bar{Z} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2}\right) \right] ).$ Using Cauchy-Schwarz inequality, we have: $D^2\left(F^{\tilde{\underline{B}}_{\alpha,\beta}}, F^{\bar{\underline{B}}_{\alpha,\beta}} \right)=\frac{1}{N} ( \text{Tr} \left[\left( W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2}\right)^2\right] + 4\text{Tr}\left[W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{Z} W_\beta \bar{Z} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2} \right]+ 4\sqrt{\text{Tr} \left[\left( W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2}\right)^2\right] \text{Tr}\left[W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{Z} W_\beta \bar{Z} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2} \right]} ).$ We show, through Von Neumann's trace inequality, that: $\text{Tr} \left[\left( W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2}\right)^2\right]\leq \alpha^2\beta^2 \text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}} \right)^2 \right],\text{and }\text{Tr}\left[W_\beta^{1/2} \bar{\bar{Z}} ^* T_\alpha \bar{Z} W_\beta \bar{Z} ^* T_\alpha \bar{\bar{Z}} W_\beta^{1/2} \right]\leq \alpha^2 \beta ^2 \sqrt{\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}} \right)^2 \right]\text{Tr}\left[\left({\bar{Z}}^*{\bar{Z}} \right)^2 \right]}.$ And, from Silverstein1995b p.185 and Assumption (b), we have: $\frac{1}{N}\text{Tr}\left[\left({\bar{Z}}^*{\bar{Z}} \right)^2 \right] \underset{n \rightarrow \infty}{\longrightarrow} c(1+c) \text{ almost surely}.$ We have also from Silverstein1995b p.185, with $K$ and $K'$ constant independent of $n$, $\mathbb{E}\left[\frac{1}{N}\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}}\right)^2 \right]\right] \leq K'\left(\mathbb{E}\left[|Z_{11}|^2 \mathrm{1}_{|Z_{11}| \geq \ln N} \right] + \frac{1}{N} \right),\mathbb{V}\left[\frac{1}{N}\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}}\right)^2 \right]\right] \leq \frac{K}{N^2} \left( \mathbb{E}\left[|Z_{11}|^4 \mathrm{1}_{|Z_{11}| \geq \sqrt{N}} \right] +1 \right).$ So, $\mathbb{E}\left[\frac{\alpha^4 \beta^4}{N}\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}}\right)^2 \right]\right] \underset{n \rightarrow \infty}{\longrightarrow} 0$ and $\mathbb{V}\left[\frac{\alpha^4 \beta^4}{N}\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}}\right)^2 \right]\right]$ is summable, so: $\frac{\alpha^4 \beta^4}{N}\text{Tr}\left[\left(\bar{\bar{Z}}^*\bar{\bar{Z}}\right)^2 \right] \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely}.$ Backing up, we have then: $D^2\left(F^{\tilde{\underline{B}}_{\alpha,\beta}}, F^{\bar{\underline{B}}_{\alpha,\beta}} \right) \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ almost surely}.$ So, $F^{\tilde{\underline{B}}_{\alpha,\beta}} - F^{\bar{\underline{B}}_{\alpha,\beta}} \overset{v}{\longrightarrow} 0$, $\overset{v}{\longrightarrow}$ denotes the vague convergence. The conclusion of that first part of the proof is that it is sufficient to show that $F^{\bar{\underline{B}}_{\alpha,\beta}} \overset{v}{\longrightarrow} \underline{F}$ for some $\underline{F} \in \mathcal{M}(\mathbb{R})$ (we will in fact prove the weak convergence) in order to prove that $F^{ \underline{B}_n} \overset{v}{\longrightarrow} \underline{F}$, which is equivalent to show that $F^{{B}_n} \overset{v}{\longrightarrow} F := \frac{1}{c} \underline{F} + \frac{1-c}{c} \mathrm{1}_{[0,\infty[}$. Moreover, as by definition $F^{B_n}(\mathbb{R}) = 1$, if $F(\mathbb{R}) = 1$, it is equivalent to prove that $F^{B_n} \underset{n \rightarrow \infty}{\implies} F$. In the following, we focus on the truncated and centralized variables. In fact, we swap: $Z_{ij}$ by $\bar{Z}_{ij}/\sqrt{\mathbb{E}[|\bar{Z}_{ij}|^2]}$,$T$ by $\mathbb{E}[|\bar{Z}_{11}|^2] T_\alpha$,$W$ by $W_\beta$. Remark that we can impose $\alpha \leq \log n$ and $\beta \leq \log n$ without compromising any of the properties on $\alpha$ and $\beta$ defined in Section 6.1.3. In addition to (a)-(e), we can now use the following Assumptions. Assume that: $\forall i,j, |Z_{ij}| \leq \log n$,$\lVert T \rVert \leq \log n$,$\lVert W \rVert \leq \log n$. From now, $B_n = \frac{1}{N} T^{1/2} Z^* W Z T^{1/2}$ and $\underline{B}_n = \frac{1}{N} W^{1/2} Z T Z^* W^{1/2}$ use the truncated variables $Z$, $T$ and $W$, and so are $F_n$, $\underline{F}_n$, $m_n$ and $\underline{m}_n$. The following aims at proving, with those truncated variables, that for some p.d.f $F$, $F^{B_n} \overset{v}{\longrightarrow} F$ a.s., which proves the theorem. For that, we prove that for some p.d.f $F$, $F^{B_n} \underset{n \rightarrow \infty}{\implies} F$ a.s. It is done using the Cauchy-Stieltjes transform: we prove that for all $z \in \mathbb{C}_+$, a.s. $m_n(z) \underset{n \rightarrow \infty}{\longrightarrow} m(z)$ where $m$ is the Cauchy-Stieltjes transform of a p.d.f. We introduce new objects of interest in the analysis, namely $q_j, r_j$, and $B_{(j)}$. For $j \in \llbracket 1,N \rrbracket$, we denote: $q_j = \frac{1}{\sqrt{n}}Z_{\cdot j}$,$r_j = \frac{1}{\sqrt{N}} T^{1/2}Z_{\cdot j} W_{jj}^{1/2}$,$B_{(j)} = B_n - r_j r_j^*$, Let's introduce a technical result. For $B$ a $(n,n)$ matrix, $q \in \mathbb{C}^n$ for which $B$ and $B + qq^*$ is invertible, we have: $q^* (B + qq^*)^{-1} = \frac{1}{1 + q^*B^{-1}q}q^* B^{-1}.$ We have: $(B_n -zI) +zI= \sum_{j=1}^N r_j r_j^*I + z(B_n - zI)^{-1}= \sum_{j=1}^N r_j r_j^* (B_n -zI)^{-1}I + z(B_n - zI)^{-1}= \sum_{j=1}^N \frac{1}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j} r_j r_j^* (B_{(j)} -zI)^{-1}c_n + z c_n m_n(z)= 1 - \frac{1}{N} \sum_{j=1}^N \frac{1}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j}\underline{m}_n(z)= - \frac{1}{N} \sum_{j=1}^N \frac{1}{z\left(1 + r_j^* (B_{(j)} -zI)^{-1}r_j\right)}.$ Here is the crucial difference between the proof in Silverstein1995a in the evenly weighted case and the weighted case. We denote: $\alpha_n = - \frac{1}{z\underline{m}_n(z)} \frac{1}{N} \sum_{j=1}^N W_{jj}\left(r_j^* (B_n -zI)^{-1}r_j - 1 \right),$ while in Silverstein1995c, $\alpha_n = 1$ is used. We have equivalently: $\alpha_n= - \frac{1}{z\underline{m}_n(z)} \frac{1}{N} \sum_{j=1}^N \frac{W_{jj}}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j}\text{and } \alpha_n= \frac{\frac{1}{N} \sum_{j=1}^N \frac{W_{jj}}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j}}{\frac{1}{N} \sum_{j=1}^N \frac{1}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j}},$ Then: $(-z \alpha_n \underline{m}_n(z) T_n - zI)^{-1} - (B_n - zI)^{-1} =(-z \alpha_n \underline{m}_n(z) T_n - zI)^{-1}\left(z \alpha_n \underline{m}_n(z) T_n + \sum_{j=1}^N r_j r_j^* \right) (B_n - zI)^{-1} =(-z \alpha_n \underline{m}_n(z) T_n - zI)^{-1}\sum_{j=1}^N \frac{r_j r_j^* (B_{(j)} -zI)^{-1} - \frac{1}{N}\alpha_n T_n (B_n -zI)^{-1}}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j}.$ Applying the trace and dividing by $n$, we obtain: $\frac{1}{n} \text{Tr}\left((-z \alpha_n \underline{m}_n(z) T_n - zI)^{-1}\right) - m_n(z) =-\frac{1}{N}\sum_{j=1}^N \frac{1}{z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)} (W_{jj}q_j^* T^{1/2} (B_{(j)} - zI)^{-1} (\alpha_n \underline{m}_n(z) T_n + I)^{-1}T^{1/2} q_j\qquad - \frac{1}{n} \text{Tr} \left( (\alpha_n \underline{m}_n(z) T_n + I)^{-1} \alpha_n T_n (B_n - zI)^{-1}\right))$ So, $\frac{1}{n} \text{Tr}\left((-z \alpha_n \underline{m}_n(z) T_n - zI)^{-1}\right) - m_n(z) =-\frac{1}{zN}\sum_{j=1}^N q_j^* \left(\frac{W_{jj}T_n^{1/2} (B_{(j)} - zI)^{-1} (\alpha_n \underline{m}_n(z) T_n + I)^{-1}T_n^{1/2}}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j} \right)q_j+ \frac{1}{zNn} \text{Tr} \left( (\alpha_n \underline{m}_n(z) T_n + I)^{-1} \sum_{j=1}^N \frac{\alpha_n T_n}{1 + r_j^* (B_{(j)} -zI)^{-1}r_j} (B_n - zI)^{-1}\right) =-\frac{1}{N} \sum_{j=1}^N \frac{1}{z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)} d_j,$ where: $d_j =W_{jj}q_j^* T_n^{1/2} (B_{(j)} - zI)^{-1} (\alpha_n \underline{m}_n(z) T_n + I)^{-1}T_n^{1/2}q_j- \frac{1}{n}\text{Tr} \left(W_{jj} (\alpha_n \underline{m}_n(z) T_n + I)^{-1} T_n (B_n - zI)^{-1}\right).$ The strategy of the proof is the following: prove that $\max_{j \leq N} d_j \longrightarrow 0$ a.s., and that $\frac{1}{n} \text{Tr}\left((-z \alpha_{n} \underline{m}_n(z) T_n - zI)^{-1}\right) - m_n(z) \longrightarrow 0$ a.s.,show that a.s. it exists $\tilde{m}(z) \in \mathbb{C} \backslash \mathbb{C}_+$ and a subsequence ${n_i}$ so that $\tilde{m}_{n_i}(z) := -z\alpha_{n_i}\underline{m}_{n_i}(z) \longrightarrow \tilde{m}(z)$ a.s.,prove that $\tilde{m}(z)$ is the unique solution in $\mathbb{C}_-$ of a functional equation, and deduce $\tilde{m}_{n}(z) \longrightarrow \tilde{m}(z)$ a.s.,deduce that a.s. it exists $m(z) \in \mathbb{C}_+$, uniquely defined in function of $\tilde{m}(z)$ so that $m_n(z) \longrightarrow m(z)$ a.s.,similarly deduce that a.s. it exists $\Theta^{(1)}(z) \in \mathbb{C}_+$, uniquely defined in function of $\tilde{m}(z)$ so that $\Theta^{(1)}_n(z) \longrightarrow \Theta^{(1)}(z)$ a.s.,conclude proving that $m$ is the Cauchy-Stieltjes transform of a p.d.f. Much of the truth of this proof relies upon the following lemma from Silverstein1995b. Let $C$ a Hermitian $n \times n$ matrix so that $\lVert C \rVert \leq 1$, and $Y = (Z_1,...,Z_n)^T$, $Z_i \in \mathbb{C}$ where the $Z_i$'s are independent, $\forall i, \mathbb{E}[Z_i] = 0$, $\mathbb{E}[|Z_i|^2] = 1$ and $|Z_i| \leq \log n$. Then, $\mathbb{E}\left[\left| Y^* C Y - \text{Tr} C \right|^6 \right] \leq K n^3 \log(n)^{12},$ where the constant $K$ does not depend on $n$, $C$, nor the distribution of $Z_1$. Lemma 3.1 Silverstein1995b assumes that the $Z_i$ are i.i.d but the "indentically distributed" assumption is in fact never used in the proof. In order to use this Lemma on $d_j$, we decompose it into negligible terms and a term of the form $q_j^* C q_j - \frac{1}{n}tr(C)$ where $C$ is independent of $q_j$. For that, we denote: $\tilde{m}_{n}(z):= -z\alpha_n\underline{m}_{n}(z)\text{and } \tilde{m}_{(j)}(z):= \frac{1}{N} \sum_{i \neq j} W_{ii}\left(r_i^* (B_{(j)} -zI)^{-1}r_i - 1 \right).$ We have the following decomposition of $d_j$: $d_j= d_j^{(1)} + d_j^{(2)} + d_j^{(3)} + d_j^{(4)},$ with: $d_j^{(1)}= W_{jj}q_j^* T_n^{1/2} (B_{(j)} - zI)^{-1} \left[\left(-\frac{\tilde{m}_n(z)}{z} T_n + I\right)^{-1} - \left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} \right]T_n^{1/2}q_j,d_j^{(2)}= W_{jj}q_j^* T_n^{1/2} (B_{(j)} - zI)^{-1}\left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1}T_n^{1/2}q_j\qquad\qquad\qquad\qquad\qquad\qquad - \frac{W_{jj}}{n}\text{Tr} \left( \left(-\frac{ \tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} T_n (B_{(j)} - zI)^{-1}\right),d_j^{(3)}= \frac{W_{jj}}{n}\text{Tr} \left( \left[\left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} - \left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} \right] T_n (B_{(j)} - zI)^{-1}\right),d_j^{(4)}= \frac{W_{jj}}{n}\text{Tr} \left(\left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} T_n \left[(B_{(j)} - zI)^{-1} - (B_{n} - zI)^{-1}\right]\right).$ In order to prove that for each $k \in \llbracket 1, 4 \rrbracket, \max_{j \leq N} d_j^{(k)} \longrightarrow 0$ a.s., we need some technical lemmas. They essentially provide the necessary inequalities to prove that $d_j^{(1)}, d_j^{(3)}$ and $d_j^{(4)}$ are indeed negligible, to finally use Lemma \ref{['lemma_rmt']} on $d_j^{(2)}$ and prove that $\max_{j \leq N} |d_j| \longrightarrow 0$ a.s. We have the following inequalities, for $j \in \llbracket 1,N \rrbracket$: $\lVert (B_n - zI)^{-1} \rVert \leq \frac{1}{v},\lVert (B_{(j)} - zI)^{-1} \rVert \leq \frac{1}{v},\frac{1}{|z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)|} \leq \frac{1}{v}.$ Let $j \in \llbracket 1, N\rrbracket$. The two first inequalities comes from the fact that $B_n$ and $B_{(j)}$ are Hermitian, so for any eigenvalue $\lambda$ of $B_n - zI$ or $B_{(j)} - zI$, we have that $|\lambda| \geq |\Im[\lambda]| = v$. For the third inequality, we remark that: $\Im r_j^* (B_{(j)}/z -I)^{-1} r_j= \frac{1}{2i} r_j^* \left[(B_{(j)}/z -I)^{-1} - (B_{(j)}/z^* -I)^{-1} \right] r_j= \frac{v}{|z|^2} r_j^* (B_{(j)}/z -I)^{-1}B_{(j)} (B_{(j)}/ z^* -I)^{-1} r_j\Im r_j^* (B_{(j)}/z -I)^{-1} r_j\geq 0.$ So, we deduce that: $\frac{1}{|z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)|} \leq \frac{1}{v}.$ We denote $\bar{W} = \frac{1}{N} \sum_{i=1}^N W_{ii}$. For $z = u +iv$, $v >0$ and $j \in \llbracket 1, N\rrbracket$, we have for any nonnegative Hermitian matrix $A$: $\left \lVert \left( - \frac{\tilde{m}_n(z)}{z} A + I \right)^{-1} \right \rVert \leq f(z,\lVert A \rVert), \text{ and } \left \lVert \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1} \right \rVert \leq f(z,\lVert A \rVert),$ where: $f(z,\lVert A \rVert) = \max \left(2, \frac{4}{v} \bar{W}\lVert A \rVert\right),\text{ if } u= 0,16 \left( \frac{|z|^2}{4v^2|u|}\bar{W} \lVert A \rVert +1 \right) \times \max \left(\frac{1}{3}, \frac{|u|}{v} \right),\text{ otherwise.}$ Let z = u + iv$, v> 0, u \in \mathbb{R}$. For $j \in \llbracket 1,N \rrbracket$, we denote by $(u_{ij})_{i=1}^n$ a set of eigenvectors of $B_{(j)}$ with associated eigenvalues $(\lambda_{ij})_{i=1}^n$. We then derive the following formulation: $R := \Re\left[- \frac{\tilde{m}_n(z)}{z} \right]= - \frac{1}{N} \sum_{j=1}^N \frac{W_{jj}}{\left|z + r_j^* \left(\frac{B_{(j)}}{z} - I \right)^{-1}r_j\right|^2} \left(u + \sum_{i=1}^n \frac{|r_j^*u_{ij}|^2}{|\lambda_{ij} - z|^2} (\lambda_{ij}u - |z|^2) \right),I :=\Im\left[- \frac{\tilde{m}_n(z)}{z} \right]= \frac{1}{N} \sum_{j=1}^N \frac{W_{jj}}{\left|z + r_j^* \left(\frac{B_{(j)}}{z} - I \right)^{-1}r_j\right|^2} \left(v + \sum_{i=1}^n \frac{|r_j^*u_{ij}|^2}{|\lambda_{ij} - z|^2} \lambda_{ij}v \right) \geq 0.$ Using Cauchy-Schwarz inequality, we deduce: $\frac{1}{N} \sum_{j=1}^N \frac{W_{jj}|z|^2\sum_{i=1}^n \frac{|r_j^*u_{ij}|^2}{|\lambda_{ij} - z|^2}}{\left|z + r_j^* \left(\frac{B_{(j)}}{z} - I \right)^{-1}r_j\right|^2} \leq\sqrt{\frac{1}{N} \sum_{j=1}^N \frac{W_{jj}\left( \sum_{i=1}^n \frac{|r_j^*u_{ij}|^2}{|\lambda_{ij} - z|^2}\right)^2}{\left|1 + r_j^* \left(B_{(j)} - zI \right)^{-1}r_j\right|^2} } \sqrt{\frac{1}{N} \sum_{j=1}^N \frac{W_{jj}}{\left|z + r_j^* \left(\frac{B_{(j)}}{z} - I \right)^{-1}r_j\right|^2}}\frac{1}{N} \sum_{j=1}^N \frac{W_{jj}|z|^2\sum_{i=1}^n \frac{|r_j^*u_{ij}|^2}{|\lambda_{ij} - z|^2}}{\left|z + r_j^* \left(\frac{B_{(j)}}{z} - I \right)^{-1}r_j\right|^2} \leq\frac{|z|\sqrt{\bar{W}}}{v\sqrt{v}} \sqrt{I}.$ So, combining both previous points, we have: $\left| R \right|\leq \frac{|u|}{v} I + \frac{|z|\sqrt{\bar{W}}}{v\sqrt{v}} \sqrt{I}.$ Suppose $u \neq 0$. We denote $K:=\frac{|z|\sqrt{\bar{W}}}{2v\sqrt{|u|}}$. We have then: $\sqrt{\frac{v}{|u|}} \left(- K + \sqrt{ K^2 + \left| R \right|}\right) \leq \sqrt{I}.$ Now, let $x \geq 0$. Then: $\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2= \left(Rx +1\right)^2 + I^2 x^2\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2\geq (-|R|x +1)^2 + \frac{v^2}{|u|^2} \left(- K\sqrt{x} + \sqrt{ K^2x + \left| R \right|x}\right)^4.$ We denote: $t := \sqrt{ K^2x +\left| R \right|x}\in \mathbb{R}_+$ . We have then: $\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2\geq (t^2 - K^2x - 1)^2 + \frac{v^2}{|u|^2} (t - K\sqrt{x})^4.$ We denote $a:=K\sqrt{x}$, $b:= \sqrt{K^2x +1}$ and we split the study of the right part of the previous equation between $[0,(a+b)/2]$ and $[(a+b)/2,+\infty[$. The lower bounds rely mainly on the fact that $b- a \geq \frac{1}{2b}$: Let $t \in [0,(a+b)/2]$. Then, $(t^2 - K^2x - 1)^2 + \frac{v^2}{|u|^2} (t - K\sqrt{x})^4\geq \left(\left(\frac{a+b}{2}\right)^2 - b^2\right)^2= \frac{(a + 3b)^2(b-a)^2}{16}\geq \frac{9(b-a)^4}{16}(t^2 - K^2x - 1)^2 + \frac{v^2}{|u|^2} (t - K\sqrt{x})^4\geq \frac{9}{16^2b^4}.$Let $t \in [(a+b)/2, +\infty[$. Then, $(t^2 - K^2x - 1)^2 + \frac{v^2}{|u|^2} (t - K\sqrt{x})^4\geq \frac{v^2}{16|u|^2}(b-a)^4(t^2 - K^2x - 1)^2 + \frac{v^2}{|u|^2} (t - K\sqrt{x})^4\geq \frac{v^2}{16^2|u|^2b^4}.$ Backing up, we have that: $\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2 \geq \frac{1}{16^2\left(K^2x +1 \right)^2}\times \min\left(\frac{v^2}{|u|^2}, 9 \right).$ It finally leads to: $\left \lVert \left( - \frac{\tilde{m}_n(z)}{z} A + I \right)^{-1} \right \rVert \leq 16 \left( \frac{|z|^2}{4v^2|u|}\bar{W} \lVert A \rVert +1 \right) \times \max \left(\frac{1}{3}, \frac{|u|}{v} \right),$ which proves the first inequality when $u \neq 0$. Now suppose $u= 0$. From Equation \ref{['sqrt_ineq']}, we have: $\left| R \right|\leq \sqrt{\frac{\bar{W}}{v}} \sqrt{I}.$ If $\bar{W}=0$, then: $\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2 =1 + x^2 I^2 \geq 1 \geq \frac{1}{4}.$ Otherwise, with $y = Rx$ and $a = \frac{v^2}{\bar{W}^4x^2}$: $\left|- \frac{\tilde{m}_n(z)}{z}x + 1 \right|^2 \geq (Rx + 1)^2 +x^2 \frac{R^4v^2}{\bar{W}^4} = ay^4 + (y+1)^2.$ Splitting the study between $]-\infty, 1/2]$ and $[1/2, +\infty[$, we have that $ay^4 + (y+1)^2 \geq \min \left(\frac{a}{16}, \frac{1}{4} \right).$ So, we deduce the first inequality when $u = 0$: $\left \lVert \left( - \frac{\tilde{m}_n(z)}{z} A + I \right)^{-1} \right \rVert \leq \max\left(2, \frac{4\bar{W}\lVert A \rVert}{v} \right).$ Let $j \in \llbracket 1, N \rrbracket$. Using the same method for $\left \lVert \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1} \right \rVert$, we have with $\bar{W}_{(j)} := \frac{1}{N} \sum_{i \neq j} W_{ii}$ : $\left| \Re\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right] \right|\leq \frac{|u|}{v} \Im\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right] + \frac{|z|\sqrt{\bar{W}_{(j)}}}{v\sqrt{v}} \sqrt{ \Im\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right]}.$ As $W_{jj} \geq 0$, $\bar{W}_{(j)} \leq \bar{W}$, so we find the same equation as Equation \ref{['sqrt_ineq']}: $\left| \Re\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right] \right|\leq \frac{|u|}{v} \Im\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right] + \frac{|z|\sqrt{\bar{W}}}{v\sqrt{v}} \sqrt{ \Im\left[- \frac{\tilde{m}_{(j)}(z)}{z} \right]}.$ So, from the previous proof of the first inequality, we deduce immediatly the second one, which complete the proof of this lemma: $u \neq 0 \implies \left \lVert \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1} \right \rVert \leq 16 \left( \frac{|z|^2}{4v^2|u|}\bar{W} \lVert A \rVert +1 \right) \times \max \left(\frac{1}{3}, \frac{|u|}{v} \right),u = 0 \implies \left \lVert \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1} \right \rVert \leq \max\left(2, \frac{4\bar{W}\lVert A \rVert}{v} \right).$ Let $j \in \llbracket 1, N \rrbracket$ and $z = u + iv, v > 0$. Then, for any matrices $A$ and $B$ of same size $n \times n$, $A$ Hermitian non-negative, we have: $\left | \text{Tr}\left[B\left(\left( - \frac{\tilde{m}_{n}(z)}{z} A + I \right)^{-1} - \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1}\right) \right]\right | \leq \left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right|n \lVert B \rVert \lVert A \rVert f(z,\lVert A \rVert)^2.$ For any invertible matrices $C_1, C_2$ of the same size than $B$, we have: $|\text{Tr}\left[B(C_1^{-1} - C_2^{-1}) \right]|= \text{Tr}\left[BC_1^{-1}(C_2-C_1)C_2^{-1} \right]|\text{Tr}\left[B(C_1^{-1} - C_2^{-1}) \right]|\leq \lVert B \rVert \times \lVert C_1^{-1} \rVert \times \lVert C_2^{-1} \rVert \times n \lVert C_2-C_1\rVert.$ From that point, the result comes immediately from Lemma \ref{['tech1']}. Let $j \in \llbracket 1, N \rrbracket$ and $z = u + iv, v > 0$. Then, for any $n \times n$ matrix $A$ Hermitian non-negative, and $r \in \mathbb{C}^n$, $\lVert r \rVert$ denoting its euclidean norm, we have: $\left | r^* \left(\left( - \frac{\tilde{m}_{n}(z)}{z} A + I \right)^{-1} - \left( - \frac{\tilde{m}_{(j)}(z)}{z} A + I \right)^{-1}\right) r\right | \leq \left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right| \lVert r \rVert^2 \lVert A \rVert f(z,\lVert A \rVert)^2.$ For any invertible matrices $C_1, C_2$ of size $n \times n$, we have: $r^*(C_1^{-1} - C_2^{-1}) r= r^*C_1^{-1}(C_2-C_1)C_2^{-1} rr^*(C_1^{-1} - C_2^{-1}) r\leq \lVert r \rVert^2 \times \lVert C_1^{-1} \rVert \times \lVert C_2^{-1} \rVert \times \lVert C_2-C_1\rVert.$ From that point, the result comes immediately from Lemma \ref{['tech1']}. Let $j \in \llbracket 1, N \rrbracket$. We denote: $A_{(j)} = \sum_{i \neq j} W_{ii} r_i r_i^*.$ Then, $\left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right| \leq \frac{2 \log n}{Nv} + \frac{\lVert A_{(j)} \rVert}{|z|vN}.$ Let $j \in \llbracket 1, N \rrbracket$. We have, using Lemma \ref{['tech0']}: $\left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right|=\left|\frac{W_{jj}r_j^*(B_n - zI)^{-1}r_j}{zN} + \sum_{i \neq j} \frac{W_{ii}}{zN} r_i^*\left[(B_n - zI)^{-1} - (B_{(j)} - zI)^{-1} \right]r_i \right|\left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right|\leq\left|\frac{W_{jj}}{zN}\left(1 - \frac{1}{1 + r_j^*(B_{(j)} - zI)^{-1}r_j} \right) \right|+ \left|\frac{1}{zN} \frac{r_j^*(B_{(j)} - zI)^{-1} A_{(j)} (B_{(j)} - zI)^{-1}r_j}{1+ r_j^* (B_{(j)} - zI)^{-1}r_j} \right|.$ For the first term, using Lemma \ref{['ineq']}, we have: $\left|\frac{W_{jj}}{zN}\left(1 - \frac{1}{1 + r_j^*(B_{(j)} - zI)^{-1}r_j} \right) \right| \leq \frac{\log n}{N|z|} + \frac{\log n}{Nv} \leq \frac{2\log n}{Nv}.$ For the second term, we have: $\left|\frac{1}{zN} \frac{r_j^*(B_{(j)} - zI)^{-1} A_{(j)} (B_{(j)} - zI)^{-1}r_j}{1+ r_j^* (B_{(j)} - zI)^{-1}r_j} \right| \leq \frac{\lVert A_{(j)}\rVert}{|z|N} \frac{\lVert (B_{(j)} - zI)^{-1}r_j \rVert^2}{|1+ r_j^* (B_{(j)} - zI)^{-1}r_j| }\left|\frac{1}{zN} \frac{r_j^*(B_{(j)} - zI)^{-1} A_{(j)} (B_{(j)} - zI)^{-1}r_j}{1+ r_j^* (B_{(j)} - zI)^{-1}r_j} \right| \leq \frac{\lVert A_{(j)}\rVert}{|z|vN}.$ Using the proof of Lemma 2.6 Silverstein1995b, we have additionally that: $\frac{\lVert (B_{(j)} - zI)^{-1}r_j \rVert^2}{|1+ r_j^* (B_{(j)} - zI)^{-1}r_j| } \leq \frac{1}{v}.$ So, $\left|\frac{1}{zN} \frac{r_j^*(B_{(j)} - zI)^{-1} A_{(j)} (B_{(j)} - zI)^{-1}r_j}{1+ r_j^* (B_{(j)} - zI)^{-1}r_j} \right| \leq \frac{\lVert A_{(j)}\rVert}{|z|vN},$ which concludes the proof. We have: $\max_{j \leq N}\lVert q_j \rVert^2 \longrightarrow 1 \text{ a.s.},\forall p \in \mathbb{N}, \max_{j \leq N} \frac{\log(n)^p}{N} \lVert A_{(j)} \rVert \longrightarrow 0 \text{ a.s.}$ The first convergence is a direct use of Lemma \ref{['lemma_rmt']} as remarked in Silverstein1995b p.338. Let $p \in \mathbb{N}$. The second convergence comes from the following derivations for $j \in \llbracket 1,N \rrbracket$: $\frac{\log(n)^{2p}}{N^2} \lVert A_{(j)} \rVert^2\leq \frac{1}{N^2} \text{Tr} \left[\left(\sum_{i \neq j} W_{ii} r_i r_i^* \right) \right]\leq \frac{c_n^2\log(n)^{2p}}{N^2} \left(\sum_{i \neq j} W_{ii}^4 \lVert T \rVert^2 \lVert q_i \rVert^4 + \sum_{i \neq j}\sum_{i' \neq i, i' \neq j} W_{ii}^2 W_{i'i'}^2| q_{i'}^* T q_i|^2 \right)\leq \frac{c_n^2\log(n)^{2p}}{N^2} \left(\sum_{i = 1}^N W_{ii}^4 \lVert T \rVert^2 \lVert q_i \rVert^4 + \sum_{i = 1}^N \sum_{i' \neq i} W_{ii}^2 W_{i'i'}^2| q_{i'}^* T q_i|^2 \right)\leq \frac{c_n^2 \log(n)^{2p+4}}{N^2} \left(\sum_{i = 1}^N \log(n)^2 \lVert q_i \rVert^4 + \sum_{i = 1}^N \sum_{i' \neq i} | q_{i'}^* T q_i|^2 \right)\frac{1}{N^2} \lVert A_{(j)} \rVert^2\leq \frac{c_n^2 \log(n)^{2p+6}}{N} \max_{i \leq N} \lVert q_i \rVert^4 + \frac{c_n^2 \log(n)^{2p+4}}{N} \max_{i \leq N} \sum_{i' \neq i} | q_{i'}^* T q_i|^2.$ The upper bound does not depend on $j$ anymore, so we have: $\max_{j \leq N} \frac{1}{N^2} \lVert A_{(j)} \rVert^2\leq \frac{c_n^2 \log(n)^{2p+6}}{N} \max_{i \leq N} \lVert q_i \rVert^4 + \frac{c_n^2 \log(n)^{2p+4}}{N} \max_{i \leq N} \sum_{i' \neq i} | q_{i'}^* T q_i|^2.$ From the first part of the proof, we have that $\max_{i \leq N} \lVert q_i \rVert^4 \longrightarrow 1 \text{ a.s.}$, so: $\frac{c_n^2 \log(n)^{2p+6}}{N} \max_{i \leq N} \lVert q_i \rVert^4 \longrightarrow 0 \text{ a.s.}$ For the second term, we have: $\frac{c_n^2 \log(n)^{2p+4}}{N} \max_{i \leq N} \sum_{i' \neq i} | q_{i'}^* T q_i|^2= \frac{c_n^2 \log(n)^{2p+4}}{N} \max_{i \leq N} \sum_{i' \neq i} q_{i'}^* (T q_i q_i^* T) q_{i'}.$ Let $i \in \llbracket 1,N \rrbracket$. We use Lemma \ref{['lemma_rmt']} in dimension $n(N-1)$ with $Y = (Z_{ki'})_{k,i'\neq i} \in \mathbb{R}^{n \times (N-1)}$ (in vectorized form) and $C = Tq_i q_i^*T(0)\ddots(0)Tq_i q_i^*T$ of size $n(N-1) \times n(N-1)$. We have then: $\mathbb{E}\left[\left|\frac{ \log(n)^{2p+4}}{N} \sum_{i' \neq i} q_{i'}^* (T q_i q_i^* T) q_{i'} - \frac{(N-1) \log(n)^{2p+4}}{nN} \lVert T q_i \rVert^2 \right|^6\right] \leq\frac{Kn^3(N-2)^3 \log\left(n(N-2)\right)^6 \log(n)^{6(2p+4)}}{N^6n^6} \lVert C \rVert^6 \leq\frac{K \log\left(nN\right)^6}{N^3n^3}\log(n)^{6(2p+8)}.$ So, for all $\epsilon > 0$: $\mathbb{P}\left[\left|\max_{i \leq N} \frac{1}{N} \sum_{i' \neq i} q_{i'}^* (T q_i q_i^* T) q_{i'} - \max_{i \leq N}\frac{N-1}{nN} \lVert T q_i \rVert^2 \right| \geq \epsilon\right] \leqN\times \mathbb{P}\left[\left|\frac{1}{N} \sum_{i' \neq 1} q_{i'}^* (T q_1 q_1^* T) q_{i'} - \frac{N-1}{nN} \lVert T q_1 \rVert^2 \right| \geq \epsilon\right] \leq\leq \frac{K \log\left(nN\right)^6}{\epsilon^6N^2n^3} \log(n)^{6(2p+8)},$ which is summable. So, $\max_{i \leq N} \frac{\log(n)^{2p+4}}{N} \sum_{i' \neq i} q_{i'}^* (T q_i q_i^* T) q_{i'} - \max_{i \leq N}\frac{(N-1)\log(n)^{2p+4}}{nN} \lVert T q_i \rVert^2 \longrightarrow 0 \text{ a.s.}$ And, from the first part of the proof: $\left| \max_{i \leq N}\frac{c_n^2(N-1)\log(n)^{2p+4}}{nN} \lVert T q_i \rVert^2 \right| \leq \frac{c_n^2 \log(n)^{2p+6}}{nN} \max_{i \leq N} \lVert q_i \rVert^2 \longrightarrow 0 \text{ a.s.}$ We can conclude the proof: $\max_{j \leq N} \frac{\log(n)^p}{N} \lVert A_{(j)} \rVert \longrightarrow 0 \text{ a.s.}$ Let $j \in \llbracket 1, N \rrbracket$. We recall that: $d_j^{(1)}= W_{jj}q_j^* T_n^{1/2} (B_{(j)} - zI)^{-1} \left[\left(-\frac{\tilde{m}_n(z)}{z} T_n + I\right)^{-1} - \left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} \right]T_n^{1/2}q_j.$ Then, using Corollary \ref{['tech1_crl2']} and Lemma \ref{['ineq']}, we have: $|d_j^{(1)}|\leq W_{jj} \lVert (B_{(j)} - zI)^{-1} \rVert \lVert T \rVert^2 \lVert q_j \rVert^2 \left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right| f(z,\log(n))^2|d_j^{(1)}|\leq \frac{\log(n)^3}{v}f(z,\log(n))^2 \lVert q_j \rVert^2 \left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right|.$ Using Lemma \ref{['tech2']}, we have: $|d_j^{(1)}|\leq \frac{\log(n)^3}{v} f(z,\log(n))^2 \left(\frac{2 \log n}{Nv} + \frac{\lVert A_{(j)} \rVert}{|z|vN}\right)\lVert q_j \rVert^2.$ By Assumption (d), $\int x dF^{W_n}(x) \longrightarrow \int x dD(x) < \infty$ a.s., i.e. $\bar{W} \longrightarrow \int x dD(x) \leq \infty$ a.s., so $\bar{W}$ is bounded a.s. As a consequence, $f(z,\log(n))^2 = O(\log(n)^2)$ a.s.. Finally, using Lemma \ref{['tech3']}, we can conclude: $\max_{j \leq N} |d_j^{(1)}| \longrightarrow 0 \text{ a.s.}$ Let $j \in \llbracket 1, N \rrbracket$. We recall that: $d_j^{(2)} =W_{jj}q_j^* T_n^{1/2} (B_{(j)} - zI)^{-1}\left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1}T_n^{1/2}q_j- \frac{W_{jj}}{n}\text{Tr} \left( \left(-\frac{ \tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} T_n (B_{(j)} - zI)^{-1}\right).$ Using Lemma \ref{['lemma_rmt']}, we have: $\mathbb{E}\left[\left|d_j^{(2)}\right|^6\right]\leq \frac{K \log(n)^{12}}{n^3} \left \lVert W_{jj} \left(-\frac{ \tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} T_n (B_{(j)} - zI)^{-1} \right \rVert^6.$ Using Lemmas \ref{['ineq']} and \ref{['tech1']}, we have: $\mathbb{E}\left[\left|d_j^{(2)}\right|^6\right]\leq \frac{K \log(n)^{24}}{v^6n^3} f(z, \log(n))^6.$ As argued above, $f(z, \log(n)) = O(\log(n))$ a.s., so $\sum_{j=1}^N \mathbb{E}\left[\left|d_j^{(2)}\right|^6\right]$ is a.s. summable. So, we can conclude: $\max_{j \leq N} |d_j^{(2)}| \longrightarrow 0 \text{ a.s.}$ Let $j \in \llbracket 1, N \rrbracket$. We recall that: $d_j^{(3)}= \frac{W_{jj}}{n}\text{Tr} \left( \left[\left(-\frac{\tilde{m}_{(j)}(z)}{z} T_n + I\right)^{-1} - \left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} \right] T_n (B_{(j)} - zI)^{-1}\right).$ Using Corollary \ref{['tech1_crl1']} and Lemma \ref{['ineq']}, we have: $|d_j^{(3)}|\leq \frac{\log(n)^3}{nv} \left|\frac{\tilde{m}_{n}(z)}{z} - \frac{\tilde{m}_{(j)}(z)}{z} \right|n f(z, \log(n))^2.$ Using Lemma \ref{['tech2']}, we have: $|d_j^{(3)}|\leq \frac{\log(n)^3}{v} \left(\frac{2 \log n}{nv} + \frac{\lVert A_{(j)} \rVert}{|z| v n}\right) f(z, \log(n))^2.$ As we argued previously, $f(z, \log(n))^2 = O(\log(n)^2)$ a.s., so from Lemma \ref{['tech3']} we can conclude: $\max_{j\ \leq N} |d_j^{(3)}|\longrightarrow 0 \text{ a.s.}$ Let $j \in \llbracket 1, N \rrbracket$. We recall that: $d_j^{(4)}= \frac{W_{jj}}{n}\text{Tr} \left(\left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} T_n \left[(B_{(j)} - zI)^{-1} - (B_{n} - zI)^{-1}\right]\right)$ Using Lemma \ref{['tech0']}, we have: $d_j^{(4)}= \frac{W_{jj}}{n} \frac{r_j^* (B_{(j)} - zI)^{-1} \left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} T (B_{(j)} - zI)^{-1}r_j}{1 + r_j^* (B_{(j)} - zI)^{-1} r_j}$ So, $|d_j^{(4)}|= \frac{\log(n)}{n} \left \lVert \left(-\frac{\tilde{m}_{n}(z)}{z} T_n + I\right)^{-1} T \right \rVert \frac{\lVert (B_{(j)} - zI)^{-1}r_j \rVert^2}{|1 + r_j^* (B_{(j)} - zI)^{-1} r_j|}$ Using the proof of Lemma 2.6 Silverstein1995b, we have that: $\frac{\lVert (B_{(j)} - zI)^{-1}r_j \rVert^2}{|1+ r_j^* (B_{(j)} - zI)^{-1}r_j| } \leq \frac{1}{v}.$ So, using Lemma \ref{['tech2']}, we have: $|d_j^{(4)}|= \frac{\log(n)^2 f(z, \log(n))}{vn}$ As argued before, $f(z, \log(n)) = O(\log(n))$ a.s., so we can conclude: $\max_{j \leq N} |d_j^{(4)}| \longrightarrow 0 \text{ a.s.}$ We can now conclude this section. The last four points prove that: $\max_{j \leq N} |d_j| \longrightarrow 0 \text{ a.s.}$ And from Lemma \ref{['ineq']}, we have for $j \in \llbracket 1, N \rrbracket$: $\frac{1}{|z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)|} \leq \frac{1}{v}.$ So, $\frac{1}{N} \sum_{j=1}^N \frac{-1}{z(1 + r_j^* (B_{(j)} -zI)^{-1}r_j)} d_j \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ a.s.}$ Using Equation \ref{['eq_cvg']}, we can now conlude that: $\frac{1}{n} \text{Tr}\left((\tilde{m}_n(z) T_n - zI)^{-1}\right) - m_n(z) \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ a.s.}$ For this section, we introduced an object used in Ledoit2009, we define: $\Theta_n^{(1)}(z) =\frac{1}{n} \text{Tr} \left((\mathrm{B}_n - zI)^{-1}T_n \right).$ Let $p \in \mathbb{N}$. We have: $\left|\tilde{m}_{n}(z) \Theta_{n}^{(1)}(z) - \left(1 + z m_{n}(z)\right)\right|= \left|\tilde{m}_{n}(z) \Theta_{n}^{(1)}(z) - \frac{1}{n} \sum_{j=1}^N \frac{r_j^*(B_{(j)}-zI)^{-1}r_j}{1+r_j^*(B_{(j)}-zI)^{-1}r_j}\right|= \left|\frac{1}{N} \sum_{j=1}^N \frac{W_{jj} \left(\Theta_{n}^{(1)}(z) - q_j^*(B_{(j)}-zI)^{-1}q_j\right)}{1+r_j^*(B_{(j)}-zI)^{-1}r_j}\right|\left|\tilde{m}_{n}(z) \Theta_{n}^{(1)}(z) - \left(1 + z m_{n}(z)\right)\right|\leq \frac{|z|}{v}\max_{j \leq N} \log(n) \left|\Theta_{n}^{(1)}(z) - q_j^*T^{1/2}(B_{(j)}-zI)^{-1}T^{1/2}q_j \right|.$ Using Lemma \ref{['lemma_rmt']}, we have that: $\mathbb{E}\left[\left|\log(n)^{p}\Theta_{n}^{(1)}(z) - \log(n)^{p}q_j^*T^{1/2}(B_{(j)}-zI)^{-1}T^{1/2}q_j \right|^6\right] \leq K \frac{\log(n)^{6(p+3)}}{n^3 v^6}.$ So, $\max_{j \leq N} \left|\log(n)^{p}\Theta_{n}^{(1)}(z) - \log(n)^{p}q_j^*T^{1/2}(B_{(j)}-zI)^{-1}T^{1/2}q_j \right| \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ a.s.}$ As $\mathbb{N}$ is countable, we have also: $\text{a.s., } \forall p \in \mathbb{N}, \max_{j \leq N} \log(n)^{p}\left|\Theta_{n}^{(1)}(z) - q_j^*T^{1/2}(B_{(j)}-zI)^{-1}T^{1/2}q_j \right| \underset{n \rightarrow \infty}{\longrightarrow} 0 \text{ a.s.}$ We can conclude: $\text{a.s., } \forall p \in \mathbb{N}, \log(n)^p \left|\tilde{m}_n(z) \Theta_{n}^{(1)}(z)- (1 + zm_n(z)) \right| \underset{n \rightarrow \infty}{\longrightarrow} 0.$ From Assumption (d), $\bar{W} \rightarrow \int xdD(x) \in \mathbb{R}_+$ a.s. We focus now on trajectories where $\bar{W} \rightarrow \int xdD(x) \in \mathbb{R}_+$, Equations \ref{['cvg']} and \ref{['theta_cvg']} hold, $F^{T_n} \implies H$ and $F^{W_n} \implies D$. Then, $(\tilde{m}_n(z))$ is bounded. Indeed, from Lemma \ref{['ineq']}, $| \tilde{m}_n(z) | \leq \bar{W} \frac{|z|}{v}$. Then, as $\Im[\tilde{m}_n(z)] \leq 0$,it exists a subsequence $\{n_i\}$ of $\mathbb{N}$ and $\tilde{m}(z) \in \mathbb{C} \backslash \mathbb{C}_+$ such that $\tilde{m}_{n_i}(z) \underset{i \rightarrow \infty}{\longrightarrow} \tilde{m}(z) \in \mathbb{C} \backslash \mathbb{C}_+$. We want to prove that: $m_{n_i}(z) - \int \frac{1}{\tau \tilde{m}(z) - z}dH(\tau) \underset{i \rightarrow \infty}{\longrightarrow} 0.$ From Equation \ref{['cvg']}, it is equivalent to prove that: $\int \frac{1}{\tau \tilde{m}_{n_i}(z) - z}dF^{T_{n_i}}(\tau) - \int \frac{1}{\tau \tilde{m}(z) - z}dH(\tau) \underset{i \rightarrow \infty}{\longrightarrow} 0.$ We prove that $\int \frac{1}{\tau \tilde{m}_{n_i}(z) - z}dF^{T_{n_i}}(\tau) \underset{i \rightarrow \infty}{\longrightarrow} -\frac{1}{z}$ using the Lebesgue's convergence theorem for weakly converging measures, as detailed in Corollary 5.1 Feinberg2019. We denote: $f: \tau \in \mathbb{R} \rightarrow \frac{1}{\tau \tilde{m}(z) - z}$ and $f_{i}: \tau \in \mathbb{R} \rightarrow \frac{1}{\tau \tilde{m}_{n_i}(z) - z}$. Regarding the hypotheses of the theorem, we have: $(f_i)_i$ is a.u.i. w.r.t. $(F^{T_{n_i}})_i$ (see (2.4) Feiberg2019 for a definition). Indeed, $\forall \tau \in \mathbb{R}_+, \forall i \in \mathbb{R}_+, |f_i(\tau)| \leq \frac{1}{v}$, so $\lim_{K \rightarrow +\infty} \limsup_{i \rightarrow +\infty} \int |f_i(\tau)| 1_{[K,+\infty[}(|f_i(\tau)|) dF^{T_{n_i}} = 0$.$F^{T_{n_i}} \implies H$ by assumption.Let $\tau \in \mathbb{R}_+$ and $\epsilon > 0$. By assumption, $\bar{W}_{n_i}$ is bounded and we denote by $\kappa$ one of its finite upper bound. By \ref{['ineq']}, we have that $\forall i, |\tilde{m}_{n_i}(z) | \leq \frac{\kappa}{|z|v}$, so $\tilde{m}(z) \leq \frac{\kappa}{|z|v}$. We define: $\delta := \min\left(1,\frac{v^2}{\frac{\kappa}{|z|v} + \tau + 1}\right) > 0.$ Then, there exists $i_0 \in \mathbb{N}$ such that $\forall i \geq i_0, |\tilde{m}_{n_i}(z) - \tilde{m}(z) | \leq \delta$. Now, let $\tau' \in ]\tau - \delta, \tau + \delta[ \cap \mathbb{R}_+$ and $i \geq i_0$. Then, $\left|f_i(\tau') - f(\tau) \right|= \frac{|\tau' \tilde{m}_{n_i}(z) - \tau \tilde{m}(z)|}{|\tau' \tilde{m}_{n_i}(z) - z|\times|\tau \tilde{m}(z) - z|}\leq \frac{1}{v^2}\left(|(\tau' - \tau) \tilde{m}(z)| + |\tau' (\tilde{m}_{n_i}(z)-\tilde{m}(z))| \right)\leq \frac{\delta}{v^2}\left(\frac{\kappa}{|z|v} + \tau + 1 \right)\left|f_i(\tau') - f(\tau) \right|\leq \epsilon.$ So $\lim_{i \rightarrow \infty, \tau' \rightarrow \tau} f_i(\tau')$ exists. So, using Corollary 5.1 Feinberg2019 on the real and imaginary parts, we deduce that $\lim_{i \rightarrow \infty} \int f_i(\tau) dF^{T_{n_i}}(\tau)$ exists and: $\lim_{i \rightarrow \infty} \int f_i(\tau) dF^{T_{n_i}}(\tau) = \int f(\tau) dH(\tau).$ It immediatly leads to: $m_{n_i}(z) - \int \frac{1}{\tau \tilde{m}(z) - z}dH(\tau) \underset{i \rightarrow \infty}{\longrightarrow} 0.$ Suppose $H = 1_{[0,+\infty[}$. We have then that a.s. $m_n(z) \underset{n \rightarrow +\infty}{\longrightarrow} -\frac{1}{z}$. Moreover, the equation $m = \int \frac{\delta}{1 + \delta c \int \frac{\tau}{\tau m - z} dH(\tau)}dD(\delta) = \int \delta dD(\delta)$ has trivially a unique solution in $\mathbb{C} \backslash \mathbb{C}_+$ that we denote $\tilde{m}^{(0)}(z) := \int \delta dD(\delta)$. As $-\frac{1}{z} = \int \frac{1}{\tau \tilde{m}^{(0)}(z) - z}dH(\tau)$, we indeed have that $m_n(z) \underset{n \rightarrow +\infty}{\longrightarrow} \int \frac{1}{\tau \tilde{m}^{(0)}(z) - z}dH(\tau)$. And $z \mapsto -\frac{1}{z}$ is the Cauchy-Stieltjes transform of the p.d.f $1_{[0,\infty[}$ which complete the proof of the Theorem in the case $H = 1_{[0,+\infty[}$. Until the end of the proof, we now suppose that $H(]0,+\infty[) > 0$. Suppose $D = 1_{[0,+\infty[}$. Then, $\bar{W}_n \underset{n \rightarrow +\infty}{\longrightarrow} 0$. By \ref{['ineq']}, $\tilde{m}_n(z) \leq \frac{\bar{W}}{|z|v}$. So the complete sequence (not the subsequence) $\tilde{m}_n(z) \underset{n \rightarrow +\infty}{\longrightarrow} 0$. From the previous section on Equation \ref{['tau']}, we proved that if $\tilde{m}_n(z)$ converges to some $\tilde{m}(z)$, then $m_n(z) - \int \frac{1}{\tau \tilde{m}(z) - z}dH(\tau) \underset{n \rightarrow \infty}{\longrightarrow} 0.$ So, with $\tilde{m}^{(1)}(z) := 0$, we have that $m_n(z) \underset{n \rightarrow \infty}{\longrightarrow} \int \frac{1}{\tau \tilde{m}^{(1)}(z) - z}dH(\tau) = - \frac{1}{z}$. Moreover, we remark that $\tilde{m}^{(1)}(z) = 0$ is the unique solution to the equation $m = \int \frac{\delta}{1 + \delta c \int \frac{\tau}{\tau m - z} dH(\tau)}dD(\delta) = \int \delta dD(\delta)$ in $\mathbb{C} \backslash \mathbb{C}_+$. And $z \mapsto -\frac{1}{z}$ is the Cauchy-Stieltjes transform of the p.d.f $1_{[0,\infty[}$ which complete the proof of the Theorem in the case $D = 1_{[0,+\infty[}$. Until the end of the proof, we now suppose that $D(]0,+\infty[) > 0$. Suppose $\tilde{m}(z) = 0$. Then, $m_{n_i}(z) \underset{i \rightarrow \infty}{\longrightarrow} - \frac{1}{z}$ and by Cauchy-Stieltjes transform property $F^{B_{n_i}} \implies 1_{[0,+\infty[}$. Consequently, $\mathbb{E}\left[\frac{1}{n_i} \text{Tr}(B_{n_i})\right] \underset{i \rightarrow +\infty}{\longrightarrow} 0$. As $\mathbb{E}\left[\frac{1}{n_i} \text{Tr}(B_{n_i})\right] = \frac{1}{n_i}\text{Tr}(T_{n_i}) \times \bar{W}_{n_i}$. $\bar{W}_{n_i}$ converges by assumption to $\int \delta dD(\delta)$, and we supposed $D(]0,+\infty[) > 0$ so $\int \delta dD(\delta) > 0$. We deduce that $\frac{1}{n_i}\text{Tr}(T_{n_i}) \underset{i \rightarrow +\infty}{\longrightarrow} 0$. So, $H = 1_{[0,+\infty[}$, which is absurd. So $\tilde{m}(z) \neq 0$. We remind that we suppose now: $H(]0,+\infty[) > 0$, $D(]0,+\infty[) > 0$ and $\tilde{m}_{n_i}(z) \underset{i \rightarrow +\infty}{\longrightarrow} \tilde{m}(z) \neq 0$. Firstly, we denote: $M_{n} := \log(n) \max_{j \leq N} \left|\Theta_{n}^{(1)}(z) - \log(n)^{p}q_j^*T^{1/2}(B_{(j)}-zI)^{-1}T^{1/2}q_j \right|$. There is $n_0$ large enough so that $\forall n \geq n_0, M_n \leq \frac{v}{2c_n}$, due to Equation \ref{['theta_cvg']}. Then, we have for $n_i \geq n_0$: $\left| \left(-c_n(1 + zm_{n_i}(z)) +1\right) - \frac{1}{N}\sum_{j=1}^N \frac{1}{1 + W_{jj} c_{n_i} \Theta_{n_i}^{(1)}} \right| =\left| \frac{1}{N}\sum_{j=1}^N \frac{1}{1 + r_j^*(B_{(j)}-zI)^{-1}r_j} - \frac{1}{1 + W_{jj} c_{n_i} \Theta_{n_i}^{(1)}} \right| \leq\frac{c_{n_i} \bar{W} M_{n_i}}{v(v - c_{n_i} M_{n_i})} \underset{i \rightarrow \infty}{\longrightarrow} 0.$ Then, we denote $\Theta^{(1)}(z) := \frac{1+zm(z)}{\tilde{m}(z)} = \int \frac{\tau}{\tau \tilde{m}(z) - z}dH(\tau)$. From Equation \ref{['theta_cvg2']}, $\Theta^{(1)}_{n_i}\underset{i \rightarrow \infty}{\longrightarrow} \Theta^{(1)}(z)$. We define: $g: \delta \in \mathbb{R} \rightarrow \frac{1}{1 + \delta c \Theta^{(1)}(z)}.$ We want to prove that: $\frac{1}{N}\sum_{j=1}^N \frac{1}{1 + W_{jj} c_{n_i} \Theta_{n_i}^{(1)}} \underset{i \rightarrow \infty}{\longrightarrow} \int g(\delta) dD(\delta).$ Let $\delta \in \mathbb{R}$. Remark that, as $\Im[\tilde{m}(z)] \leq 0$: $\Im[\Theta^{(1)}(z)] = \Im\left[\frac{1+zm(z)}{\tilde{m}(z)} \right] = \Im \left[\int \frac{\tau}{\tau \tilde{m}(z) - z}dH(\tau) \right] \geq v\int \frac{\tau}{|\tau \tilde{m}(z) - z|^2}dH(\tau) > 0.$ So, from Silverstein1995c p.338, we have: $|g(\delta)| \leq \frac{|\Theta^{(1)}(z)|}{\Im[\Theta^{(1)}(z)]}.$ And, for $n_i$ large enough so that $\forall n_i \geq n_1, \Im[\Theta^{(1)}_{n_i}] > 0$: $\left| \frac{1}{1 + \delta c_{n_i} \Theta_{n_i}^{(1)}} - g(\delta)\right| \leq \frac{1}{c_{n_i} \Im[\Theta_{n_i}^{(1)}(z)]}\times \frac{|\Theta^{(1)}(z)|}{\Im[\Theta^{(1)}(z)]} \left|c\Theta^{(1)}(z) - c_{n_i} \Theta^{(1)}_{n_i}\right| .$ Using \ref{['cvg_delta']}, \ref{['ineq_delta1']} and \ref{['ineq_delta2']}, we have that: $c_{n_i}\left(1 + zm_{n_i}(z)\right) - 1 \underset{i \rightarrow \infty}{\longrightarrow} \int \frac{ 1}{1 + \delta c \Theta^{(1)}(z)}dD(\delta).$ We conclude that: $1 + zm_{n_i}(z) \underset{i \rightarrow \infty}{\longrightarrow} \int \frac{ \delta \Theta^{(1)}(z)}{1 + \delta c \Theta^{(1)}(z)}dD(\delta).$ Finally, using \ref{['tau']} and \ref{['delta']}, we have that: $\tilde{m}(z) = \int \frac{\delta}{1 + \delta c \int \frac{\tau}{\tau \tilde{m}(z) - z} dH(\tau)}dD(\delta).$ In this section, we show that there is at most one solution $m \in \mathbb{C} \backslash \mathbb{C}_+$ so that: $m = \int \frac{\delta}{1 + \delta c \int \frac{\tau}{\tau m - z} dH(\tau)}dD(\delta).$ Let $m \in \mathbb{C} \backslash \mathbb{C}_+$, verifying \ref{['fun_eq']}. Then, as $H(]0, +\infty[) > 0$: $\Im[m]= -\int \frac{\delta^2 c \int \frac{\tau(\tau \Im[-m] + v)}{|\tau m - z|^2}dH(\tau)}{\left| 1 + \delta c \int \frac{\tau}{\tau m - z}dH(\tau) \right|^2} dD(\delta) < 0.$ So, with $\mathbb{C}_- = \{ y \in \mathbb{C} | \Im[y] < 0\}$: $\left[m \in \mathbb{C} \backslash \mathbb{C}_+ \text{ verifies }\ref{['fun_eq']}\text{ and } H(]0, +\infty[) > 0\right] \iff \left[m \in \mathbb{C}_- \text{ verifies }\ref{['fun_eq']}\text{ and } H(]0, +\infty[) > 0\right].$ Let $m_1 = u_1 - i v_1 \in \mathbb{C}_-$ and $m_2 = u_2 - i v_2 \in \mathbb{C}_-$ solving \ref{['fun_eq']} at $z = u + iv \in \mathbb{C}_+$. Then, $m_1 - m_2= \int \frac{\delta^2 c \left(\int \frac{\tau}{\tau m_2 - z} dH(\tau) - \int \frac{\tau}{\tau m_1 - z} dH(\tau) \right)}{\left(1 + \delta c \int \frac{\tau}{\tau m_1 - z} dH(\tau)\right)\left(1 + \delta c \int \frac{\tau}{\tau m_2 - z} dH(\tau)\right)} dD(\delta)m_1 - m_2= (m_1 - m_2) \times \int \frac{\delta^2 c \int \frac{\tau^2}{(\tau m_1 - z)(\tau m_2 - z)} dH(\tau) }{\left(1 + \delta c \int \frac{\tau}{\tau m_1 - z} dH(\tau)\right)\left(1 + \delta c \int \frac{\tau}{\tau m_2 - z} dH(\tau)\right)} dD(\delta)$ Using Hölder inequality on the last term and using \ref{['uni_imag']} at the end, we have: $\left| \int \frac{\delta^2 c \int \frac{\tau^2}{(\tau m_1 - z)(\tau m_2 - z)} dH(\tau) }{\left(1 + \delta c \int \frac{\tau}{\tau m_1 - z} dH(\tau)\right)\left(1 + \delta c \int \frac{\tau}{\tau m_2 - z} dH(\tau)\right)} dD(\delta) \right|^2 \leq\left| \int \frac{\delta^2 c \int \frac{\tau^2}{|\tau m_1 - z|^2} dH(\tau) }{\left|1 + \delta c \int \frac{\tau}{\tau m_1 - z} dH(\tau)\right|^2} dD(\delta) \right| \times \left|\int \frac{\delta^2 c \int \frac{\tau^2}{|\tau m_2 - z|^2} dH(\tau) }{\left|1 + \delta c \int \frac{\tau}{\tau m_2 - z} dH(\tau)\right|^2} dD(\delta) \right| <\left| \int \frac{\delta^2 c \int \frac{\tau^2 + \tau \frac{v}{v_1}}{|\tau m_1 - z|^2} dH(\tau) }{\left|1 + \delta c \int \frac{\tau}{\tau m_1 - z} dH(\tau)\right|^2} dD(\delta) \right| \times \left|\int \frac{\delta^2 c \int \frac{\tau^2+ \tau \frac{v}{v_2}}{|\tau m_2 - z|^2} dH(\tau) }{\left|1 + \delta c \int \frac{\tau}{\tau m_2 - z} dH(\tau)\right|^2} dD(\delta) \right| =\left|\frac{v_1}{v_1} \right| \times \left|\frac{v_2}{v_2} \right| = 1.$ Remark that the inequality is strict because we supposed $H(]0, +\infty[) > 0$. Injecting this inequation in \ref{['uni_diff']}, we find: $|m_1 - m_2| \neq 0 \implies |m_1 - m_2| < |m_1 - m_2|.$ So there is at most one solution $m \in \mathbb{C} \backslash \mathbb{C}_+$ verifying \ref{['fun_eq']}. Backing up, we proved that almost surely, $\tilde{m}_n(z)$ is bounded and every convergent subsequence of $\tilde{m}_n(z)$ converge towards the unique $\tilde{m}(z) \in \mathbb{C} \backslash \mathbb{C}_+$ verifying \ref{['tilde_m']}. So, a.s., $\tilde{m}_n(z) \underset{n \rightarrow \infty}{\longrightarrow} \tilde{m}(z)$. We also proved that, almost surely, if $\tilde{m}_n(z) \underset{n \rightarrow \infty}{\longrightarrow} \tilde{m}(z)$ then: $m_n(z) \underset{n \rightarrow \infty}{\longrightarrow} m(z) = \int \frac{1}{\tau \tilde{m}(z) - z} dH(\tau)$,$\Theta^{(1)}_n(z) \underset{n \rightarrow \infty}{\longrightarrow} \Theta^{(1)}(z) = \int \frac{\tau}{\tau \tilde{m}(z) - z} dH(\tau)$. So, almost surely, $m_n(z) \underset{n \rightarrow \infty}{\longrightarrow} m(z) = \int \frac{1}{\tau \tilde{m}(z) - z} dH(\tau)$ where $\tilde{m}(z)$ is the unique solution in $\mathbb{C} \backslash \mathbb{C}_+$ to \ref{['tilde_m']}. The last point to prove is that $m$ is Stieltjes transform of a p.d.f. As pointwise limit of Stieltjes transform, it is enough to prove that, using Theorem 7 Najim2016, $i y m(iy) \underset{y \rightarrow \infty}{\longrightarrow} -1$ to show that $m$ is a Stieltjes transform of a p.d.f. Consider a trajectory where $\bar{W}$ is bounded, say by $\kappa$, and $\tilde{m}_n(z) \rightarrow \tilde{m}(z)$. Then, $\forall z \in \mathbb{C}_+, | \tilde{m}_n(z)| \leq \kappa \frac{|z|}{v}$. So, $\forall y \in \mathbb{R}_+^*, |\tilde{m}(iy)| \leq \kappa$. Consequently, $\left | -i y m(iy) - 1 \right| = \left | \int \frac{-\tau \tilde{m}(iy)}{\tau \tilde{m}(iy) - iy}dH(\tau) \right| \leq \frac{\kappa}{y}.$ So, $i y m(iy) \underset{y \rightarrow \infty}{\longrightarrow} -1$, which proves that $m$ is a Stieltjes transform a p.d.f. and concludes the proof. Mohsen Pourahmadi. Covariance estimation: The glm and regularization perspectives. Statistical Science, 26(3):369 -- 387, 2011.Jianqing Fan, Yuan Liao, and Han Liu. An overview of the estimation of large covariance and precision matrices. The Econometrics Journal, 19(1):C1--C32, 2016.Yuan Ke, Stanislav Minsker, Zhao Ren, Qiang Sun, and Wen-Xin Zhou. User-friendly covariance estimation for heavy-tailed distributions, 2019.Olivier Ledoit and Michael Wolf. The power of (non-)linear shrinking: A review and guide to covariance matrix estimation. Journal of Financial Econometrics, 20(1):187--218, 06 2020.Clifford Lam. High-dimensional covariance matrix estimation. WIREs Computational Statistics, 12(2):e1485, 2020.V A Marčenko and L A Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik, 1(4):457, apr 1967.J. W. Silverstein. Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices. Journal of Multivariate Analysis, 55(2):331--339, 1995.J.W. Silverstein and Z.D. Bai. On the empirical distribution of eigenvalues of a class of large dimensional random matrices. Journal of Multivariate Analysis, 54(2):175--192, 1995.Jack W. Silverstein and Sang Il Choi. Analysis of the limiting spectral distribution of large dimensional random matrices. Journal of Multivariate Analysis, 54:295--309, 1995.Z. D. Bai and Jack W. Silverstein. No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. The Annals of Probability, 26(1):316 -- 345, 1998.Z. D. Bai and Jack W. Silverstein. CLT for linear spectral statistics of large-dimensional sample covariance matrices. The Annals of Probability, 32(1A):553 -- 605, 2004.Taras Bodnar, Nestor Parolya, and Frederik Veldman. Nonlinear shrinkage test on a large-dimensional covariance matrix. Statistica Neerlandica, 79, 07 2024.Romain Couillet, Mérouane Debbah, and Jack Silverstein. A deterministic equivalent for the analysis of correlated mimo multiple access channels. Information Theory, IEEE Transactions on, 57:3493 -- 3514, 07 2011.Nicolas Honnorat and Mohamad Habes. Covariance shrinkage can assess and improve functional connectomes. NeuroImage, 256:119229, 2022.Joel Bun, Romain Allez, Jean-Philippe Bouchaud, and Marc Potters. Rotational invariant estimator for general noisy matrices. IEEE Transactions on Information Theory, 62(12):7475--7490, December 2016.Christian Bongiorno and Damien Challet. Covariance matrix filtering and portfolio optimisation: the average oracle vs non-linear shrinkage and all the variants of dcc-nls, 2023.A. B. D. Monvel, Alexei M. Khorunzhy, and V. Vasilchuck. Limiting eigenvalue distribution of random matrices with correlated entries. 1996.Antonia Tulino and Sergio Verdú. Random matrix theory and wireless communications. Foundations and Trends in Communications and Information Theory, 1, 01 2004.Zdzisław Burda, Jerzy Jurkiewicz, and Bartłomiej Wacław. Spectral moments of correlated wishart matrices. Physical Review E, 71(2), February 2005.Lixin Zhang. Spectral Analysis of large dimensional random matrices. PhD thesis, National University of Singapore, 2007.Debashis Paul and Jack W. Silverstein. No eigenvalues outside the support of the limiting empirical spectral distribution of a separable covariance matrix. Journal of Multivariate Analysis, 100(1):37--57, 2009.Michael Fleermann and Werner Kirsch. Proof methods in random matrix theory, 2022.Eugene A. Feinberg, Pavlo O. Kasyanov, and Yan Liang. Fatou's lemma in its classic form and lebesgue's convergence theorems for varying measures with applications to mdps, 2019.Olivier Ledoit and Sandrine Péché. Eigenvectors of some large sample covariance matrix ensembles, 2009.Romain Couillet and Walid Hachem. Analysis of the limiting spectral measure of large random matrices of the separable covariance type, 2015.Y.Q. Yin. Limiting spectral distribution for a class of random matrices. Journal of Multivariate Analysis, 20(1):50--68, 1986.