Concentration inequalities for strong laws and laws of the iterated logarithm

TL;DR

This paper derives non-asymptotic, time-uniform concentration inequalities for sums $S_n=\sum_{i=1}^n X_i$ of i.i.d. mean-zero variables, yielding concrete, finite-sample generalizations of classical limit theorems such as the SLLN and LIL. It first establishes $L^1$ concentration bounds using truncated first moments $\mathsf U_\mathsf P(x)$, then extends to $L^q$ bounds for $q\in[1,2)$ with an exponentially decaying term in $m$ and a truncated moment term $\mathsf U_\mathsf P^{(q)}(\cdot)$, enabling distribution-uniform SLLNs (Chung) and Marcinkiewicz–Zygmund SLLNs. The results yield non-asymptotic iterated-logarithm inequalities akin to Darling–Robbins, along with self-normalized and distribution-uniform LILs, and further provide Baum–Katz-type non-asymptotic bounds. The work also develops game-/pathwise analogues via $\mathsf P$-$e$-processes and a composite Ville framework for families of measures, linking uniform convergence properties to explicit, verifiable probabilistic certificates with practical non-asymptotic guarantees.

Abstract

We derive concentration inequalities for sums of independent and identically distributed random variables that yield non-asymptotic generalizations of several strong laws of large numbers including some of those due to Kolmogorov [1930], Marcinkiewicz and Zygmund [1937], Chung [1951], Baum and Katz [1965], Ruf, Larsson, Koolen, and Ramdas [2023], and Waudby-Smith, Larsson, and Ramdas [2024]. As applications, we derive non-asymptotic iterated logarithm inequalities in the spirit of Darling and Robbins [1967], as well as pathwise (sometimes described as "game-theoretic") analogues of strong laws and laws of the iterated logarithm.

Theorems & Definitions (7)

• Theorem 2.1: $L^1$ concentration for the strong law of large numbers
• Corollary 2.2: Chung's distribution-uniform $L^1$ SLLN chung_strong_1951
• proof : Proof of \ref{['theorem:l1-concentration']}
• Theorem 2.3: $L^q$ concentration for the strong law of large numbers
• Corollary 2.4: Marcinkiewicz-Zygmund strong laws of large numbers marcinkiewicz1937fonctions
• proof
• Corollary 2.5: Distribution-uniform $L^q$ SLLNs for $q \in [1, 2)$ chung_strong_1951waudby2024distribution] Let $q \in [1,2)$ and let $\mathcal{P} \subset \overline \mathcal{P}$ be a family of probability measures for which the random variable $X$ has a $\mathcal{P}$-uniformly integrable $q^{\text{th}}$ moment, meaning $\lim_{{x \nearrow \infty}} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf E_\mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q \geq x \} \right ] = 0.$ The SLLN holds uniformly in $\mathcal{P}$ at a rate of $o(n^{1/q-1})$ meaning that for any $\varepsilon > 0$, $\lim_{m \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{k^{1/q}} \geq \varepsilon \right ] = 0.$ It is easy to see how \ref{['corollary:wlr-slln']} follows from \ref{['theorem:lq-concentration']} by taking a supremum over $\mathcal{P}$ and a limit as ${m \nearrow \infty}$ on both the left-hand and right-hand sides of the inequality \ref{['eq:lq-concentration']}. Again, note that by waudby2024distribution, uniform integrability of the $q^{\text{th}}$ moment is necessary and sufficient to conclude \ref{['eq:wsetal']} so \ref{['theorem:lq-concentration']} is sharp in the sense that it yields pointwise and uniform Marcinkiewicz-Zygmund-type SLLNs under the same moment assumptions. In the following section, we consider the case of $q=2$ and arrive at an iterated logarithm inequality that can be used to derive the upper bound in Kolmogorov's law of the iterated logarithm. While the former sections provided inequalities that can be used to deduce that $n^{-1}S_n = o(n^{1/q-1})$ with $\mathsf{P}$-probability one for $q \in [1,2)$, such a deduction is not possible when $q = 2$ as exemplified by the law of the iterated logarithm which states that if $\sigma_\mathsf{P}^2 := \mathrm{Var}_\mathsf{P}[X_1] < \infty$, then $\limsup_{{n \nearrow \infty}}\frac{S_n / \sigma_\mathsf{P}}{\sqrt{2 n \log \log n}} = 1\qquad\text{with $\mathsf{P}$-probability one.}$ Juxtaposing the non-asymptotic results of the previous section with the asymptotic statement in \ref{['eq:kolmogorov-lil']} naturally motivates the question: Do there exist non-asymptotic time-uniform concentration inequalities for sums of random variables with iterated logarithm rates of convergence? It appears that this question was first posed and partially solved by darling1967iterateddarling1968some. A simplified version of darling1967iterated states that if $(X_n)_{n \in \mathbb N}$ are i.i.d. and $\widebar \sigma$-sub-Gaussian, meaning that $\mathsf E_\mathsf{P} \left [ e^{t |X_1| } \right ] \leq e^{ t^2 {\widebar \sigma^2} / 2 }; \qquad t \in \mathbb R,$ then for any $m \in \mathbb N \setminus \{1\}$ and any $\varepsilon > 0$, $\mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / \widebar \sigma|}{\sqrt{k (2(1+\varepsilon)^2 \log \log k + 2(1+\varepsilon) \log 2 )}} \geq 1 \right ] \leq \frac{\log_{1+\varepsilon}^{-\varepsilon}(m)}{\varepsilon }.$ It is easy to check that the above implies that $\limsup_{{n \nearrow \infty}} |S_n/\sigma| / \sqrt{2 \widebar n \log \log n} \leq 1$ with $\mathsf{P}$-probability one, resembling the behavior of \ref{['eq:kolmogorov-lil']} but with $\sigma_\mathsf{P}^2$ replaced by the sub-Gaussian variance proxy $\widebar \sigma^2$. Several other iterated logarithm inequalities exist --- sometimes referred to as "finite laws of the iterated logarithm" --- such as in darling1968somebalsubramani2014sharpjamieson2014bestkaufmann2016complexityzhao2016adaptive, and howard_exponential_2018. What all of these iterated logarithm inequalities including those of darling1967iterateddarling1968some have in common, however, is that they require the underlying random variables to be sub-Gaussian or at the very least, have finite moment generating functions. Meanwhile, the law of the iterated logarithm in \ref{['eq:kolmogorov-lil']} is only a statement about random variables in $L^2$ that need not have any finite moment higher than a variance. A partial exception to this rule exists in howard_exponential_2018, where the authors derive a sub-Gaussian iterated logarithm inequality with a variance proxy taking the form of a convex combination of bounds on the variance and the squared random variables themselves. We will use this inequality in our proof of \ref{['theorem:lil']}. Nevertheless, the aforementioned iterated logarithm inequalities do not directly yield the upper bound in the LIL under only a finite second moment assumption nor do they yield distribution-uniform generalizations of Kolmogorov's LIL. The following inequalities will serve precisely this purpose. Fix $\mathsf{P} \in \overline \mathcal{P}$, let $\widebar \sigma \in (0, \infty)$, and assume that $X_1$ has finite variance $\mathrm{Var}_\mathsf{P}[X_1] \leq {\widebar \sigma^2}$. Define $\widebar \mathsf{U}_\mathsf{P}^{(2)}(x) := \mathsf E_\mathsf{P}[|X_1^2 - \sigma_\mathsf{P}^2| \mathds{1} \{ |X_1^2 - \sigma_\mathsf{P}^2| \geq x \}];\ x \geq 0.$ Then for any $\varepsilon > 0$, $\lambda \in (0, 1/2)$, and $m \in \mathbb N \setminus \{1\}$, we have \mathsf{P} \left [ {\sup_{k \geq m}} \frac{ |S_k/ \widebar \sigma|}{c_\varepsilon \sqrt{k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ] \leq \frac{\log_{1+\varepsilon}^{-\varepsilon}(m/3)}{\varepsilon \zeta(1+\varepsilon) } + \frac{262}{(\varepsilon^2 \widebar \sigma^4) \land 1} \left ( m^{2\lambda - 1} + \widebar \mathsf{U}_\mathsf{P}^{(2)}(m^\lambda) \right ), where $c_\varepsilon := ( (1+\varepsilon)^{5/4} + (1+\varepsilon)^{3/4} ) / \sqrt{2}$, $\ell_\varepsilon := \log(2 \zeta(1+\varepsilon) / \log(1 + \varepsilon))$, and the function $\zeta$ is the Riemann zeta function, given by $\zeta(z) := \sum_{j=1}^\infty j^{-z};\ z > 1$. The proof of \ref{['theorem:lil']} can be found in \ref{['proof:lil']}. Applying \ref{['theorem:l1-concentration', 'theorem:lil']} together, we have the following studentized analogue of the above inequality with ${\widebar \sigma^2}$ replaced by the sample variance. Fix $\mathsf{P} \in \overline \mathcal{P}$ and assume that $X_1$ has finite variance $\sigma_\mathsf{P}^2 := \mathrm{Var}_\mathsf{P}[X_1] < \infty$. Define the truncated normalized second moment $\widehat{\mathsf{U}}_\mathsf{P}^{(2)}(x)$ by $\widehat{\mathsf{U}}_\mathsf{P}^{(2)}(x) := \mathsf E_\mathsf{P} \left [ \frac{X_1^2}{\sigma_\mathsf{P}^2} \mathds{1} \left \{ \frac{X_1^2}{\sigma_\mathsf{P}^2} \geq x \right \} \right ];\quad x \geq 0,$ and for each $n \in \mathbb N$ the sample variance $\widehat{\sigma}_n^2$ by $\widehat{\sigma}_n^2 := \frac{1}{n} \sum_{i=1}^n (X_i - \widehat{\mu}_n)^2,\quad\text{where}\quad \widehat{\mu}_n := \frac{1}{n} \sum_{i=1}^n X_i.$ Then for any $\varepsilon > 0$, $\lambda \in (0,1/2)$, and $m \in \mathbb N \setminus\{1\}$, \mathsf{P} \left [ {\sup_{k \geq m}} \frac{| S_k/ \widehat{\sigma}_k|}{c_\varepsilon \sqrt{ (1+2\varepsilon) k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ] \leq \frac{\log_{1+\varepsilon}^{-\varepsilon}(2m/3)}{\varepsilon \zeta(1+\varepsilon)} + \frac{786}{\varepsilon^2 \land 1} \left ( m^{2\lambda-1} + \widehat{\mathsf{U}}_\mathsf{P}^{(2)}(m^\lambda) \right ), where $c_\varepsilon$ and $\ell_\varepsilon$ are as in \ref{['theorem:lil']}. The proof of \ref{['corollary:self-normalized-lil']} can be found after that of \ref{['theorem:lq-concentration']} in \ref{['proof:lil']}. As applications of \ref{['theorem:lil']} and \ref{['corollary:self-normalized-lil']}, we have the following distribution-uniform generalizations of the upper bound in Kolmogorov's law of the iterated logarithm, which seem to be new to the literature. Let $\mathcal{P} \subset \overline \mathcal{P}$. If the second moment of $X_1$ is $\mathcal{P}$-uniformly integrable, i.e. $\sup_{\mathsf{P} \in \mathcal{P}} \mathsf{U}_\mathsf{P}^{(2)}(x) \searrow 0$ as ${x \nearrow \infty}$, then there exists $\widebar \sigma \in (0, \infty)$ so that $\sup_{\mathsf{P} \in \mathcal{P}} \mathrm{Var}_\mathsf{P}[X_1] \leq \widebar \sigma^2$ and the law of the iterated logarithm holds uniformly in $\mathcal{P}$, meaning that for any $\delta > 0$ we have $\lim_{m \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / {\widebar \sigma}| }{\sqrt{2 k \log \log k}} \geq 1 + \delta \right ] = 0.$ Furthermore, if the normalized random variable $X_1 / \sigma_\mathsf{P}$ has a $\mathcal{P}$-uniformly integrable second moment, i.e. $\sup_{\mathsf{P} \in \mathcal{P}} \widehat{\mathsf{U}}_\mathsf{P}^{(2)}(x) \searrow 0$ as ${x \nearrow \infty}$, where $\widehat{\mathsf{U}}_\mathsf{P}^{(2)}(x)$ is defined in \ref{['eq:studentized second moment']}, then for any $\delta > 0$, $\lim_{m \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / \widehat{\sigma}_k |}{\sqrt{2 k \log \log k}} \geq 1 + \delta \right ] = 0.$ Clearly, \ref{['eq:uniform-lil-variance']} implies the upper bound of Kolmogorov's LIL in \ref{['eq:kolmogorov-lil']} since taking $\mathcal{P} = \{ \mathsf{P} \}$ and $\widebar \sigma = \sigma_\mathsf{P}$ for any distribution $\mathsf{P}$ for which $\sigma_\mathsf{P}^2 < \infty$ yields for any $\delta > 0$, $\mathsf{P} \left [ \limsup_{n \nearrow \infty} \frac{|S_n / \sigma_\mathsf{P}|}{\sqrt{2 n \log \log n}} \geq 1 + \delta \right ] = \lim_{m \nearrow \infty} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / \sigma_\mathsf{P}|}{\sqrt{2 k \log \log k}} \geq 1 + \delta \right ] = 0.$ Nevertheless, \ref{['corollary:uniform-lil']} (and by extension, \ref{['theorem:lil']} and \ref{['corollary:self-normalized-lil']}) contain additional details about those distributional properties to which the asymptotics of the LIL are uniform. In particular, \ref{['corollary:uniform-lil']} can be viewed as an extension of Chung's $L^1$ uniform SLLN discussed in \ref{['corollary:chung-slln']} to random variables in $L^2$. Let us now observe how the concentration inequalities of the previous sections can be used to derive SLLNs and LILs in the spirit of baum1965convergence (see also neri2025quantitative) and in fact strengthen some of (the forward implications in) their results. For a probability measure $\mathsf{P} \in \overline \mathcal{P}$ and any $q \in [1,2)$, baum1965convergence show that $\mathsf E_\mathsf{P}[|X_1|^q \log( |X_1| + 1 )] < \infty \quad \text{if and only if}\quad \forall \varepsilon > 0,~~ \sum_{m=1}^\infty \frac{1}{m} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{k^{1/q}} \geq \varepsilon \right ] < \infty.$ The series being finite allows one to conclude that the probability in the summand vanishes at a sufficiently fast rate $r_m$ so that $r_m / m$ is summable, while the SLLNs of Kolmogorov and marcinkiewicz1937fonctions provide no such rate of convergence. However, note that the Baum-Katz SLLN described above is neither stronger nor weaker than the SLLNs of Kolmogorov and marcinkiewicz1937fonctions since the stronger conclusion above requires finiteness of a logarithmically higher moment. Furthermore, one cannot directly conclude the distribution-uniform SLLNs of chung_strong_1951 and waudby2024distribution from the result of baum1965convergence alone even when assuming a uniformly bounded higher moment. As we will see in the following proposition, the concentration inequality in \ref{['theorem:lq-concentration']} is sufficiently sharp to provide an explicit upper bound on the infinite series in \ref{['eq:baum-katz']}, culminating in a strengthening and alternative proof of their result. Let $\mathsf{P} \in \overline \mathcal{P}$ and $q \in [1,2)$. For any $m \in \mathbb N$ and $\varepsilon > 0$, define $P_m^{(\varepsilon)} := \mathsf{P} [ {\sup_{k \geq m}} |k^{-{1/q}} S_k| \geq \varepsilon ]$. Then, $\sum_{m=1}^\infty \frac{P_m^{(\varepsilon)}}{m} \leq 1 + \frac{c_q}{e \log(2^{1/q-1/2})} + \frac{2603 \left ( \mathsf E_\mathsf{P}[|X_1|^q \log (38|X_1|^q / \varepsilon^q + 1)] \right ) }{(2-q) (\varepsilon^2 \land 1)},$ where $c_q := 2/(2-q)$. In particular, if $\mathcal{P} \subset \overline \mathcal{P}$ is a collection of probability measures for which $\sup_{{\mathsf{P} \in \mathcal{P}}}\mathsf E_\mathsf{P} [|X_1|^q \log (|X_1|^q + 1)] < \infty,$ then $\sup_{{\mathsf{P} \in \mathcal{P}}}\sum_{m=1}^\infty P_m^{(\varepsilon)} / m < \infty$ for any $\varepsilon > 0$. Note that \ref{['corollary:baum-katz']} can be viewed as a strengthening of the forward implication in \ref{['eq:baum-katz']} since an explicit upper bound on the series $\sum_{m=1}^\infty P_m^{(\varepsilon)} / m$ is provided. The proof is short so we provide it here. First, note that \sum_{m=1}^\infty \frac{P_m^{(\varepsilon)}}{m}= \sum_{j=1}^\infty \sum_{m=2^{j-1}}^{2^j-1} \frac{P_{m}^{(\varepsilon)}}{m} \leq \sum_{j=1}^\infty \frac{\cancel{2^{j-1}}}{\cancel{2^{j-1}}} \mathsf{P} \left [ \sup_{k \geq 2^{j-1}} \frac{ |S_k|}{k} \geq \varepsilon \right ] \leq 1 + \sum_{j=1}^\infty P_{2^j}^{(\varepsilon)}. Applying \ref{['theorem:lq-concentration']} yields \sum_{j=1}^\infty P_{2^j}^{(\varepsilon)} \leq c_{q} \sum_{j=1}^\infty \exp \left ( -2^{j(1/q - 1/2)} \right ) + \frac{451}{\varepsilon^2 \land 1} \sum_{j=1}^\infty \mathsf E_\mathsf{P} \left [ \frac{|X_1|^q}{38} \mathds{1} \{ |X_1|^q \geq \varepsilon^q 2^{j(1/2 - q/4)} \} \right ]. Analyzing the series in the second term of the right-hand side, we have \sum_{j=1}^\infty \mathsf E_\mathsf{P} \left [\frac{ |X_1|^q}{38} \mathds{1} \{ |X_1|^q \geq \varepsilon^q 2^{j(1/2 - q/4)} \} \right ]= \mathsf E_\mathsf{P} \left [|X_1|^q \sum_{j=1}^\infty \mathds{1} \{ 38|X_1|^q \varepsilon^{-q} \geq 2^{j(1/2 - q/4)} \} \right ]\leq \frac{1}{\log(2)} \mathsf E_\mathsf{P} \left [ |X_1|^q \frac{ \log(38 |X_1|^q \varepsilon^{-q} + 1)}{1/2 - q/4} \right ], where the inequality upper bounds the number of non-zero indicators in the series. Since $\log (2) \geq 0.69314$, we have $\sum_{m=1}^\infty \frac{P_m^{(\varepsilon)}}{m} \leq 1 + c_q \sum_{j=1}^\infty \exp \left ( -2^{j(1/q - 1/2)} \right ) + \frac{2603 \left ( \mathsf E_\mathsf{P}[|X_1|^q \log (38|X_1|^q \varepsilon^{-q} + 1)] \right ) }{(2-q) (\varepsilon^2 \land 1)}.$ Now, observe that \sum_{j=1}^\infty \exp \left ( -2^{j(1/q-1/2)} \right )\leq \int_{0}^\infty \exp \left ( -2^{y(1/q-1/2)} \right )\mathrm{d} y = \frac{1}{\log(2^{1/q - 1/2})}\int_1^\infty x^{-1} e^{-x} \mathrm{d} x \leq \frac{1}{e \log(2^{1/q - 1/2})}, where the equality uses the change of variables $x = 2^{y(1/q - 1/2)}$ and the last inequality bounds $x^{-1}$ by $1$ inside the integral. This completes the proof of the upper bound in \ref{['eq:baum-katz-type-upperbound']}. Next, to prove that $\sup_{\mathsf{P} \in \mathcal{P}} \mathsf E_\mathsf{P}[|X_1|^q \log(|X_1|^q + 1)] < \infty$ implies $\sup_{\mathsf{P} \in \mathcal{P}} \sum_{m=1}^\infty P_m^{(\varepsilon)} / m < \infty$, notice that \mathsf E_\mathsf{P}\left[|X_1|^q \log \left(\frac{38 |X_1|^q}{\varepsilon^q} + 1\right)\right]\leq \mathsf E_\mathsf{P}\left [|X_1|^q \log \left ( \frac{38}{\varepsilon^q \land 1} (|X_1|^q + 1) \right ) \right ]= \log\left(\frac{38}{\varepsilon^q \land 1}\right) \mathsf E_\mathsf{P} \left [ |X_1|^q \right ] + \mathsf E_\mathsf{P} \left [|X_1|^q \log (|X_1|^q + 1) \right ]; hence the statement follows. Let us now consider an analogous setup to \ref{['corollary:baum-katz']} but for the LIL. Recall that in baum1965convergence, the authors show that for a random variable $X_1$ with unit variance, if $\mathsf E_\mathsf{P}[|X_1|^2 \log^{1+\delta} (|X_1| + 1)] < \infty$ for some $\delta > 0$, then $\text{for any } \gamma > 0,\quad \sum_{m=3}^\infty \frac{1}{m \log(m)} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{\sqrt{2 k \log \log k}} \geq 1 + \gamma \right ] < \infty.$ The following result employs \ref{['theorem:lil']} to obtain a distribution-uniform Baum-Katz-type LIL under weaker moment conditions than those listed above. Fix $\mathsf{P} \in \overline \mathcal{P}$. For $\varepsilon > 0$, denote $P_m^{(\varepsilon)} := \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{c_\varepsilon \sqrt{k [\log \log ((1 + \varepsilon)^2 k ) + \ell_\varepsilon]}} \geq 1 \right ],$ where the constants $c_\varepsilon$ and $\ell_\varepsilon$ are given as in \ref{['theorem:lil']}. Then for any $m \in \mathbb N \setminus \{1\}$, any $\varepsilon > 0$, and any $\delta > 0$, \sum_{m=2}^\infty \frac{P_m^{(\varepsilon)}}{m \log (m)} \leq \sum_{m=2}^\infty \frac{\log^\varepsilon(1 + \varepsilon) \varepsilon^{-1} \zeta(1 + \varepsilon)}{m \log^{1 + \varepsilon}(2m/3)} + \frac{262}{\varepsilon^2} \left ( \sum_{m=2}^\infty \frac{m^{-4/3}}{ \log (m)} + \sum_{m=2}^\infty \frac{ 1 + \mathsf E_\mathsf{P}[X_1^2 \log^\delta (X_1^2 + 1)]}{(1/3)^{\delta}m \log^{1+\delta}(m)} \right ). In particular, if $\mathcal{P} \subset \overline \mathcal{P}$ is a collection of probability measures for which $X_1$ has unit variance and $\sup_{\mathsf{P} \in \mathcal{P}} \mathsf E_\mathsf{P}[X_1^2 \log^\delta (X_1^2 + 1)] < \infty$ for some $\delta > 0$, then $\sup_{\mathsf{P} \in \mathcal{P}} \sum_{m=2}^\infty \frac{P_m^{(\varepsilon)}}{m \log (m)} < \infty.$ Applying \ref{['theorem:lil']} with $\lambda = 1/3$, we have \sum_{m=2}^\infty \frac{1}{m\log m}P_m^{(\varepsilon)}\leq \sum_{m=2}^\infty \frac{1}{m\log m} \left ( \frac{1}{\varepsilon \zeta(1+\varepsilon) \log^{\varepsilon}_{1+\varepsilon}(2m/3) } + \frac{262}{\varepsilon^2 } \left ( m^{-1/3} + \mathsf E_\mathsf{P}[|X_1^2 - 1| \mathds{1} \{ |X_1^2-1| > m^{1/3} \}] \right )\right )\leq \sum_{m=2}^\infty \frac{\log_{1+\varepsilon}^{-\varepsilon}(2m/3)}{m \log(m) \varepsilon \zeta(1+\varepsilon)} + \frac{262}{\varepsilon^2} \sum_{m=2}^\infty \left(\frac{m^{-4/3}}{ \log(m)} + \frac{\mathsf E_\mathsf{P} \left [ (X_1^2 + 1) \mathds{1} \{ X_1^2 + 1 > m^{1/3} \} \right ]}{m \log m} \right ). Now, \ref{['eq:251009']} follows by observing that for any $a,b \geq 1$, we have $\mathds{1} \{ a > b \} \leq \log^\delta(a) / \log^\delta(b)$. Even in the case where $\mathcal{P} = \{ \mathsf{P} \}$ is taken to be a singleton, \ref{['proposition:baum-katz-lil']} improves on baum1965convergence by only requiring that $\mathsf E_\mathsf{P}[X_1^2 \log^\delta(X_1^2 + 1)] < \infty$ for some $\delta > 0$ rather than for some $\delta > 1$. Furthermore, note that similar to the relationship between \ref{['corollary:baum-katz']} and the SLLNs of Kolmogorov, Marcinkiewicz, and Zygmund, baum1965convergence requires a stronger moment assumption than Kolmogorov's LIL. Nevertheless, the inequality in \ref{['theorem:lil']} is sharp enough to deduce both. While SLLNs and LILs are typically written in terms of probability-one events such as in \ref{['eq:intro-kolmogorov-slln']} and \ref{['eq:kolmogorov-lil']}, there has been renewed interest in pathwise (or game-theoretic) presentations and proofs of almost sure limit theorems that rely on the explicit construction of so-called $e$-processes, the definition of which we review now. Fix $\mathsf{P} \in \overline \mathcal{P}$ and let $(\mathcal{F}_n)_{n \in \mathbb N_0}$ be a filtration. A nonnegative $(\mathcal{F}_n)_{n \in \mathbb N_0}$-adapted stochastic process $(E_n)_{n \in \mathbb N_0}$ is said to be a $\mathsf{P}$-$e$-process if $\mathsf E_\mathsf{P}[E_\tau] \leq 1$ for an arbitrary $(\mathcal{F}_n)_{n \in \mathbb N_0}$-stopping time $\tau$. The above process is said to be a $\mathcal{P}$-$e$-process for an arbitrary family of probability measures $\mathcal{P}$ if \ref{['eq:e-proc']} holds for all ${\mathsf{P} \in \mathcal{P}}$. Broadly speaking, given an event $A \in \mathcal{F}$, a proof of the claim "$\mathsf{P}[A] = 0$" is often given the description of pathwise or game-theoretic if one constructs an explicit $\mathsf{P}$-$e$-process $(E_n)_{n \in \mathbb N_0}$ with the property that this process diverges pathwise on $A$, meaning that $E_n(\omega) \nearrow \infty$ for every $\omega \in A$; see e.g. sasai2019erdos and ruf2022composite. Such a construction is directly connected to the notion of $A$ having probability zero as illustrated by Ville's theorem ville1939etude which states that $\mathsf{P}[A] = 0$ if and only if there exists a $\mathsf{P}$-$e$-process that diverges pathwise on $A$. In Ville's writing of his theorem, the $e$-process was to be interpreted as the accumulated wealth of a hypothetical gambler playing a "fair" sequential game. Intuitively, a gambler playing such a game over time can never become infinitely rich except with zero probability; formally, $\mathsf{P}[\sup_{n \in \mathbb N} E_n < \infty] = 1$, a consequence of Ville's inequality for nonnegative supermartingales ville1939etude (see also howard_exponential_2018 for an elementary proof) applied to the Snell envelope of $(E_n)_{n \in \mathbb N}$ under $\mathsf{P}$. It is because of this hypothetical gambler and the game they are playing that such proofs and constructions are often described as "game-theoretic". However, the same phrase is also used to describe theorems and proofs in the so-called game-theoretic formalism of probability as set out by shafer2005probabilityshafer2019game, where Kolmogorov's axioms of measure-theoretic probability are eschewed. To emphasize that we are operating in a purely measure-theoretic setting, we drop the term "game-theoretic" altogether going forward and use the term "pathwise" instead. One will typically find Ville's theorem stated in terms of a nonnegative $\mathsf{P}$-martingale diverging to $\infty$ pathwise on an event $A$ rather than a $\mathsf{P}$-$e$-process doing so, but the former can be replaced by the latter without loss of generality; see ruf2022composite. Note that all nonnegative $\mathsf{P}$-martingales started at one are $\mathsf{P}$-$e$-processes --- a consequence of Doob's optional stopping theorem --- but there exist $e$-processes that are neither supermartingales nor martingales. Let us now illustrate how pathwise proofs of SLLNs and LILs can be directly derived once provided access to the concentration inequalities of \ref{['section:slln', 'section:lil']}. First, fix $q \in [1,2)$ and consider a probability measure $\mathsf{P} \in \overline \mathcal{P}$ so that $\mathsf E_\mathsf{P}|X_1|^q < \infty$. Consider also the event $A_{q\mathrm{\text{-}div}}$ which states that the SLLN does not hold at the Marcinkiewicz-Zygmund rate of $o(n^{1/q-1})$: $A_{q\mathrm{\text{-}div}} := \left \{ \frac{S_n}{n^{1/q}} \text{ does not converge to } 0 \right \}.$ Construct the process $(E_n^{(q)})_{n \in \mathbb N_0}$ by $E_n^{(q)} := \sum_{j \in \mathbb N} \mathds{1} \left \{ \max_{m_j \leq k \leq n} \frac{|S_k|}{k^{1/q}} \geq \frac{1}{j} \quad\text{and}\quad n \geq m_j \right \}.$ Here, we use the notation $m_j := \min \left \{ m \in \mathbb N : \frac{2\exp (-m^{1/q - 1/2})}{2-q} + 451 j^2 \mathsf{U}_\mathsf{P}^{(q)}\left(\frac{m^{1/2 - q/4}}{38 j^q}\right) \leq 2^{-j} \right \}; \qquad j \in \mathbb N,$ where $\mathsf{U}_\mathsf{P}^{(q)}$ is as in \ref{['theorem:lq-concentration']}. To see why $E_n^{(q)}$ forms a $\mathsf{P}$-$e$-process that diverges pathwise on $A_{q\mathrm{\text{-}div}}$, first note that for any stopping time $\tau$, we have $\mathsf E_\mathsf{P} \left [ E_\tau^{(q)} \right ] \leq \sum_{j \in \mathbb N} \mathsf{P} \left [ \bigcup_{k \geq m_j} \left\{ \frac{|S_k|}{k^{1/q}} \geq \frac{1}{j} \right\} \right ] \leq \sum_{j\in \mathbb N} 2^{-j},$ where the second inequality follows from \ref{['theorem:lq-concentration']} instantiated with $\varepsilon = 1/j$. It follows that $E_n^{(q)}$ forms a $\mathsf{P}$-$e$-process. Let us now see why $E_n(\omega) \nearrow \infty$ as ${n \nearrow \infty}$ for every $\omega \in A_{q\mathrm{\text{-}div}}$. Notice that by definition of $A_{q\mathrm{\text{-}div}}$, for every $\omega \in A_{q\mathrm{\text{-}div}}$ there exists some $T(\omega) \in \mathbb N$ so that for every $j \geq T(\omega)$, we have $|S_k(\omega)|/k^{1/q} \geq 1/j$ for infinitely many $k \in \mathbb N$. Therefore, the $j^{\text{th}}$ summand in \ref{['eq:e-process-qdiv']} is equal to one eventually, and therefore $E_n^{(q)}(\omega)$ diverges as ${n \nearrow \infty}$. A similar story can be told for the LIL. Consider some $\mathsf{P} \in \overline \mathcal{P}$ for which $\sigma_\mathsf{P}^2 := \mathrm{Var}_\mathsf{P}[X_1] < \infty$ and define the event $A_{\mathrm{fluc}}$ for which the LIL does not hold by $A_{\mathrm{fluc}} := \left \{ \limsup_{n \nearrow \infty} \frac{|S_n/ \sigma_\mathsf{P}|}{\sqrt{2 n \log \log n}} > 1 \right \},$ and the process $(E_n^{(2)})_{n \in \mathbb N_0}$ by $E_n^{(2)} := \sum_{j\in\mathbb N} \mathds{1} \left \{ \max_{m_j \leq k \leq n} \frac{|S_k / \sigma_\mathsf{P}|}{c_{1/j} \sqrt{k [ \log \log ((1 + 1/j)^2 k) + \ell_{1/j}]}} \geq 1 \quad\text{and}\quad n \geq m_j \right \},$ and $E_0^{(2)} := 0$ where $m_j$ is the smallest integer for which the right-hand side of the inequality in \ref{['theorem:lil']} instantiated with $(\lambda, \varepsilon, \widebar \sigma^2) = (1/3, 1/j, \sigma_\mathsf{P}^2)$ is at most $2^{-j}$, i.e. $m_j := \min \left \{ m \in \mathbb N \setminus \{1\} : \frac{j\log^{-1/j}_{1+1/j}(2m/3)}{\zeta(1 + 1/j)} + \frac{262}{(\sigma_\mathsf{P}^4 / j^2)\land 1} \left ( m^{-1/3} + \widebar \mathsf{U}_\mathsf{P}^{(2)} (m^{1/3}) \right ) \leq 2^{-j} \right \},$ where $c_{1/j}$, $\ell_{1/j}$, $\zeta$, and $\widebar \mathsf{U}_\mathsf{P}$ are as in \ref{['theorem:lil']}. The justification for why $(E_n^{(2)})_{n \in \mathbb N_0}$ is both a $\mathsf{P}$-$e$-process and diverges pathwise on $A_{\mathrm{fluc}}$ is essentially the same as above but with \ref{['theorem:lil']} invoked instead of \ref{['theorem:lq-concentration']}. As far as we know, $(E_n^{(q)})_{n \in \mathbb N_0}$ and $(E_n^{(2)})_{n \in \mathbb N_0}$ are the first $e$-processes to be derived for the SLLN with Marcinkiewicz-Zygmund rates and for the LIL only under finite $q^{\text{th}}$ moment assumptions when $q \in (1,2]$. The case of $q=1$ under a finite first moment assumption was completed in ruf2022composite and the case of the LIL was studied in sasai2019erdos for self-normalized martingales. The discussion thus far has focused on $\mathsf{P}$-$e$-processes for a single probability measure $\mathsf{P}$. We will provide an analogue of Ville's theorem for a family of probability measures $\mathcal{P}$ when applied to events that can be represented as union-intersections of certain lattices of events. Concretely, we will consider events $A$ that can be written as $A = \bigcup_{\varepsilon > 0} \bigcap_{m=1}^\infty A^{(m,\varepsilon)}$ for some collection $(A^{(m,\varepsilon)})_{(m,\varepsilon) \in \mathbb N \times \mathbb R^+}$ with the property that $A^{(m_1, \varepsilon_1)} \supseteq A^{(m_2, \varepsilon_2)}$ for every $(m_1, \varepsilon_1), (m_2, \varepsilon_2) \in \mathbb N\times \mathbb R^+$ whenever $m_1 \leq m_2$ and $\varepsilon_1 \leq \varepsilon_2$. The lattice structure is induced from the aforementioned set inclusion. For brevity, we will refer to such collections as "event lattices". The reason for considering events of this type is because distribution-uniform SLLNs and LILs are implicitly statements about the lattices that represent events rather than the events per se. For example, recall that Chung's SLLN (\ref{['corollary:chung-slln']}) states that if the first moment is $\mathcal{P}$-uniformly integrable, then $\forall \varepsilon > 0,\quad \lim_{m \nearrow \infty} \sup_{\mathsf{P} \in \mathcal{P}} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{k} \geq \varepsilon \right ] = 0.$ Note that $A_{\mathrm{1\text{-}div}}$ can be written as a union-intersection of the events in the above probabilities, i.e. $A_{\mathrm{1\text{-}div}} = \left \{ \frac{S_n}{n} \text{ does not converge to 0} \right \} = \bigcup_{\varepsilon > 0} \bigcap_{m=1} \left \{ {\sup_{k \geq m}} \frac{|S_k|}{k} \geq \varepsilon \right \}.$ Similarly, the distribution-uniform SLLN of waudby2024distribution (\ref{['corollary:wlr-slln']}) is a statement about the lattice given by $A^{(m,\varepsilon)} := \{{\sup_{k \geq m}} |S_k| / k^{1/q} \geq \varepsilon\}$, and the distribution-uniform LIL upper bound in \ref{['corollary:uniform-lil']} is one about the lattice given by $A^{(m,\varepsilon)} := \{ {\sup_{k \geq m}}|S_k/\sigma| / \sqrt{2 \widebar k \log \log k} \geq 1 + \varepsilon \}$ where $\widebar \sigma$ is defined in the statement of the corollary. Relevant to the present section, we will soon see that the event lattices used to represent "distribution-uniform convergence" in the sense of chung_strong_1951 and as seen in \ref{['eq:chung-again']} are crucial to a uniform generalization of $e$-processes diverging to $\infty$, and that these two notions of uniformity are equivalent in a certain sense. Before making this connection explicit, we need the following definition. Let $(E_n)_{n \in \mathbb N_0}$ be a process. We say that $(E_n)_{n \in \mathbb N_0}$ diverges pathwise to $\infty$ tail-uniformly on an event lattice $(A^{(m,\varepsilon)})_{(m,\varepsilon)\in \mathbb N \times \mathbb R^+}$ if for all $\varepsilon > 0$, $\lim_{m \nearrow \infty} \inf_{\omega \in A^{(m, \varepsilon)}} \sup_{n \in \mathbb N} E_n(\omega) = \infty.$ \ref{['definition:tail event-uniformity']} rules out those processes that may diverge pathwise to $\infty$ for all $\omega \in A^{(m,\varepsilon)}$ for every $m \in \mathbb N$ and $\varepsilon > 0$ but may not do so uniformly in this sequence of tail events as ${m \nearrow \infty}$. Concretely, if a process does not diverge pathwise tail-uniformly, then there could exist some constant $U$ so that no matter what values $\varepsilon>0$ and $m \in \mathbb N$ are taken to be, there may exist some $\omega_{m,U} \in A^{(m,\varepsilon)}$ depending on $m$ and $U$ for which $\sup_{n \in \mathbb N} E_n(\omega_{m,U}) \leq U$. In \ref{['section:implications-composite']}, we give an example of an $e$-process that diverges pathwise on the event that the SLLN fails to hold, but not tail-uniformly on a natural lattice that approximates that event. As alluded to previously, the reason for introducing \ref{['definition:tail event-uniformity']} is due to its role in equivalent characterizations of strong asymptotic events having distribution-uniform probability zero in the sense of chung_strong_1951. This role is made precise in the following result. Fix a family of probability measures $\mathcal{P}$, a filtration $(\mathcal{F}_n)_{n \in \mathbb N_0}$, and the event lattice consisting of unions of events: $\mathcal{A} := \left ( \bigcup_{k\geq m} A_k^{(\varepsilon)} \right )_{(m , \varepsilon) \in \mathbb N \times \mathbb R^+},$ where $(A_k^{(\varepsilon)})_{(k,\varepsilon) \in \mathbb N \times \mathbb R^+}$ is a collection of sets satisfying $A_k^{(\varepsilon)} \in \mathcal{F}_k$ for all $(k,\varepsilon) \in \mathbb N \times \mathbb R^+$. Then $\forall \varepsilon > 0,\quad \lim_{m \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ \bigcup_{k \geq m} A_k^{(\varepsilon)} \right ] = 0$ if and only if there exists an $(\mathcal{F}_n)_{n \in \mathbb N_0}$-adapted $\mathcal{P}$-$e$-process $(E_n)_{n \in \mathbb N_0}$ that diverges pathwise to $\infty$ tail-uniformly on $\mathcal{A}$. Note that \ref{['proposition:distribution-uniform-ville']} can be stated irrespective of any set $A$ that uses $\mathcal{A}$ as a representing event lattice. The proof of \ref{['proposition:distribution-uniform-ville']} is short and constructive so we present it here. Suppose that \ref{['eq:uniform-convergence-abstract']} holds. For any $j \in \mathbb N$, define $m_j := \min \left \{ m \in \mathbb N : {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ \bigcup_{k \geq m} A_k^{(1/j)} \right ] \leq 2^{-j} \right \},$ and for any $\omega \in \Omega$ and $n \in \mathbb N_0$, define $E_n(\omega) := \sum_{j \in \mathbb N} \mathds{1} \left \{ \omega \in \bigcup_{k = m_j}^n A_k^{(1/j)} \right \}.$ To show that $(E_n)_{n \in \mathbb N_0}$ is a $\mathcal{P}$-$e$-process, let $\tau$ be any $(\mathcal{F}_n)_{n \in \mathbb N_0}$-stopping time and observe that \mathsf E_\mathsf{P} \left [ E_\tau \right ]\leq \sum_{j \in \mathbb N} \mathsf{P} \left [ A_\tau^{(1/j)} \mathds{1} \{ \tau \geq m_j \} \right ] \leq \sum_{j \in \mathbb N} \mathsf{P} \left [ \bigcup_{k \geq m_j} A_k^{(1/j)} \right ] \leq 1 by construction of $m_j$. We next argue that $(E_n)_{n \in \mathbb N_0}$ diverges pathwise to $\infty$ tail-uniformly on $\mathcal{A}$. To this end, let $\varepsilon > 0$ and $U > 0$ be arbitrary. Define $j(\varepsilon) := \left \lceil 1/\varepsilon \right \rceil$ and consider an arbitrary $m \geq m_{j(\varepsilon) + U}$. Then for any $\omega \in A^{(\varepsilon, m)} = \bigcup_{k\geq m} A_k$, we have \sup_{n \in \mathbb N} E_n(\omega)\geq \sum_{j=j(\varepsilon)}^{j(\varepsilon) + U} \mathds{1} \left \{ \omega \in \bigcup_{k = m_j}^\infty A_k^{(1/j)} \right \} \geq \sum_{j=j(\varepsilon)}^{j(\varepsilon) + U} \mathds{1} \left \{ \omega \in \bigcup_{k = m_j}^\infty A_k^{(\varepsilon)} \right \} = U + 1, where the second inequality follows from monotonicity of $A_k^{(\varepsilon)}$ in $\varepsilon > 0$ and the final equality follows from the fact that $\bigcup_{k \geq m} A_k^{(\varepsilon)} \subseteq \bigcup_{k \geq m_j} A_k^{(\varepsilon)}$ for every $j \leq j(\varepsilon) + U$. Since $U > 0$ was arbitrary, we have $\lim_{m \nearrow \infty} \inf_{\omega \in A^{(\varepsilon, m)}} \sup_{n \in \mathbb N} E_n(\omega) = \infty.$ We now prove the converse. Suppose that $(E_n)_{n \in \mathbb N_0}$ diverges pathwise to $\infty$ tail-uniformly on $\mathcal{A}$. By \ref{['definition:tail event-uniformity']}, we have that for every $\varepsilon> 0$ and $U > 0$, there exists some $m_U$ for which $\sup_{n \in \mathbb N}E_n(\omega) \geq U$ for all $\omega \in A^{(\varepsilon, m_U)}$. Considering the stopping time $\tau := \min \{n \in \mathbb N_0: E_n \geq U\}$ we get \sup_{\mathsf{P} \in \mathcal{P}} \mathsf{P} \left [ \bigcup_{k = m_U}^\infty A_k^{(\varepsilon)} \right ]\leq \sup_{\mathsf{P} \in \mathcal{P}} \mathsf{P} \left [ \sup_{n \in \mathbb N} E_n \geq U \right ] = {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ E_\tau \geq U \right ] \leq \frac{1}{U} \sup_{\mathsf{P} \in \mathcal{P}} \mathsf E_\mathsf{P} \left [ E_\tau \right ] \leq \frac{1}{U}. Since $U > 0$ was arbitrary, this completes the proof. Now that we have \ref{['proposition:distribution-uniform-ville']} in place, we are ready to use it to provide necessary and sufficient conditions for a uniform pathwise SLLN to hold. Fix $q \in [1, 2)$ and let $A_{q\mathrm{\text{-}div}}$ be the event that the SLLN does not hold at the Marcinkiewicz-Zygmund rate of $o(n^{1/q - 1})$. Notice that this event has an event lattice presentation as follows: $A_{q\mathrm{\text{-}div}} = \left \{ \lim_{n \nearrow \infty} \frac{S_n}{n^{1/q}} \neq 0 \right \} \equiv \bigcup_{\varepsilon > 0} \bigcap_{m=1}^\infty \bigcup_{k=m}^\infty \left \{ \frac{|S_k|}{k^{1/q}} \geq \varepsilon \right \}.$ We now have the following near-immediate corollary. Let $q \in [1, 2)$ and $\mathcal{P} \subset \overline \mathcal{P}$. Let $(\mathcal{F}_n)_{n \in \mathbb N_0}$ be the filtration generated by $(X_n)_{n \in \mathbb N}$ and define the event lattice $\mathcal{A}_{q\mathrm{\text{-}div}} := \left ( \bigcup_{k\geq m}\left \{ \frac{|S_k|}{k^{1/q}} \geq \varepsilon \right \} \right )_{(m, \varepsilon) \in \mathbb N \times \mathbb R^+}.$ Then the following three conditions are equivalent: $X_1$ has a uniformly integrable $q^{\text{th}}$ moment: $\lim_{x \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf E_\mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q \geq x \} \right ] = 0.$The SLLN holds uniformly in $\mathcal{P}$ at a rate of $o(n^{1/q - 1})$, meaning that $\forall \varepsilon > 0,\quad \lim_{m \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k|}{k^{1/q}} \geq \varepsilon \right ] = 0.$There exists a $\mathcal{P}$-$e$-process that diverges pathwise to $\infty$ tail-uniformly on $\mathcal{A}_{q\mathrm{\text{-}div}}$. The first equivalence $(i) \iff (ii)$ follows from waudby2024distribution and the second $(ii)\iff(iii)$ follows from \ref{['proposition:distribution-uniform-ville']}. Let us now move on to the case of $q=2$, where ruf2022composite prompted the future direction of "[extending] game-theoretic constructions for the law of the iterated logarithm to [uniform] settings". The following corollary provides one answer to this inquiry. Let $\mathcal{P} \subset \overline \mathcal{P}$ be a family of probability measures for which $X_1$ has a uniformly bounded variance, i.e., ${\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathrm{Var}_\mathsf{P} [X_1] \leq \widebar \sigma^2 < \infty$. Let $(\mathcal{F}_n)_{n \in \mathbb N_0}$ be the filtration generated by $(X_n)_{n \in \mathbb N}$ and define the event lattice $\mathcal{A}_{\mathrm{fluc}}$ describing super-iterated-logarithm fluctuations: $\mathcal{A}_{\mathrm{fluc}} := \left ( \left\{ \frac{|S_n/\widebar \sigma|}{\sqrt{2 n \log \log n} } \geq 1 + \varepsilon \right\} \right )_{(n, \varepsilon) \in \mathbb N \times \mathbb R^+},$ recognizing that $A_{\mathrm{fluc}} := \bigcup_{\varepsilon > 0} \bigcap_{m\geq 1} \bigcup_{k\geq m}\{ |S_k /\widebar \sigma|/ \sqrt{2 k \log \log k} \geq 1 + \varepsilon \}$ is the converse of the upper bound in the LIL. Then $\forall \varepsilon > 0,\quad \lim_{m \nearrow \infty} \sup_{\mathsf{P} \in \mathcal{P}} \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / \widebar \sigma|}{\sqrt{2 k \log \log k}} \geq 1 + \varepsilon \right ] = 0$ if and only if there exists a $\mathcal{P}$-$e$-process that diverges pathwise tail-uniformly on $\mathcal{A}_{\mathrm{fluc}}$. The above follows immediately from \ref{['proposition:distribution-uniform-ville']}. While there do exist (non-uniform) pathwise LILs in the literature such as those of sasai2019erdos, they require more than 2 moments in the i.i.d. case and hence are not viewed as pathwise counterparts of the upper bound in Kolmogorov's LIL. A so-called "composite" generalization of Ville's theorem for families of probability measures was recently introduced to the literature by ruf2022composite. Their generalization of a zero-probability event to a class of measures $\mathcal{P}$ is given in terms of the inverse capital outer measure $\nu_\mathcal{P}$ defined for any $A \in \mathcal{F}_\infty$, where $(\mathcal{F}_n)_{n \in \mathbb N_0}$ is some filtration, by $\nu_\mathcal{P}[A] := \inf_{\tau \in \mathcal{T} : A \subseteq \{\tau < \infty \}} \sup_{\mathsf{P} \in \mathcal{P}} \mathsf{P} [\tau < \infty],$ where $\mathcal{T}$ is the set of all $(\mathcal{F}_n)_{n \in \mathbb N_0}$-stopping times. As suggested by the name, $\nu_\mathcal{P}$ is not a measure but an outer measure. In short, ruf2022composite states that for an event $A \in \mathcal{F}$, $\nu_\mathcal{P}[A] = 0$ if and only if there exists a $\mathcal{P}$-$e$-process diverging to $\infty$ pathwise on $A$. Such a result differs from \ref{['proposition:distribution-uniform-ville']} since the latter requires that the $e$-process additionally diverges pathwise to $\infty$ tail-uniformly. We illustrate the gap between these two notions of divergence in the context of the SLLN and demonstrate that tail-uniformity is a strictly stronger notion than pathwise divergence. In ruf2022composite, the authors prove a composite SLLN which states that if $\lim_{x \nearrow \infty} {\sup_{\mathsf{P} \in \mathcal{P}} \,}\mathsf{U}_\mathsf{P}(x) = 0$, then there exists a $\mathcal{P}$-$e$-process that diverges pathwise to $\infty$ on $A_{\mathrm{1\text{-}div}}$, equivalently, $\nu_\mathcal{P}[A_{\mathrm{1\text{-}div}}] = 0$. The authors conjecture that "some condition like [uniform integrability] is necessary to restrict $\mathcal{P}$" in order to conclude that $\nu_\mathcal{P}[A_{\mathrm{1\text{-}div}}]= 0$. We now demonstrate that uniform integrability is not a necessary condition by constructing a family $\mathcal{P}^\star$ for which $X_1$ is not $\mathcal{P}^\star$-uniformly integrable alongside a $\mathcal{P}^\star$-$e$-process that diverges on $A_{\mathrm{1\text{-}div}}$. Indeed, for each $b \in \mathbb N$, let $\mathsf{P}_b$ be the probability measure so that $X_1$ takes the values $\pm b$ with equal $\mathsf{P}_b$-probability. In other words, $\mathsf{P}_b[X_1 = x] = (1/2)^{\mathds{1} \{ x \in \{-b,b\} \}}$ for any $x$. Letting $\mathcal{P}^\star = \{ \mathsf{P}_b : b \in \mathbb N \}$, we see that $X_1$ is not $\mathcal{P}^\star$-uniformly integrable since for any $x \geq 0$, $\mathsf E_{\mathsf{P}_b} [|X_1| \mathds{1} \{ |X_1| \geq x \}] = b$ if $b \geq x$ and $0$ otherwise. Therefore, $\sup_{b \in \mathbb N}\mathsf E_{\mathsf{P}_b} [|X_1| \mathds{1} \{ |X_1| \geq x \}] = \infty$ for any $x \geq 0$. Nevertheless, consider the process $E_n^\star := \sum_{j\in \mathbb N} \mathds{1} \left \{ \max_{m_j \leq k \leq n} \frac{|S_k|}{k|X_1|} \geq \frac{1}{j} \right \},$ where $m_j := \min \{ m \in \mathbb N \setminus \{1\} : 262 j^2 (m^{-1/3} + \mathsf{U}_{\mathsf{P}_1}(m^{1/3}) ) \leq 2^{-j} \}; \qquad j \in \mathbb N,$ for any $b >0$ and where $\mathsf{U}_{\mathsf{P}_1}$ is defined in \ref{['theorem:l1-concentration']}. Since $|S_k|/|X_1|$ under $\mathsf{P}_b$ has the same distribution as $|S_k|$ under $\mathsf{P}_1$, and by the same arguments as in \ref{['SS:5.1']}, but with \ref{['theorem:lq-concentration']} replaced by \ref{['theorem:l1-concentration']}, $(E_n^\star)_{n \in \mathbb N_0}$ is a $\mathcal{P}^\star$-$e$-process. Alternatively, one can view $(E_n^\star)_{n \in \mathbb N_0}$ as "waiting" to see $X_1$, at which point $\mathsf{P}_b$ conditionally on $|X_1|$ equals $\mathsf{P}_{|X_1|}$ and is known exactly. Hence the composite behavior of ${\sup_{k \geq m}} |S_k| / k$ under $\mathcal{P}^\star$ can be reduced to pointwise behavior of $|S_k| / (k|X_1|)$ under $\mathsf{P}_1$. The event $A_{\mathrm{1\text{-}div}}$ can be represented through two different event lattices: $A_{\mathrm{1\text{-}div}} = \bigcup_{\varepsilon > 0} \bigcap_{m=1} \left \{ {\sup_{k \geq m}} \frac{|S_k|}{k} \geq \varepsilon \right \} = \bigcup_{\varepsilon > 0} \bigcap_{m=1} \left \{ {\sup_{k \geq m}} \frac{|S_k|}{k|X_1|} \geq \varepsilon \right \}.$ Noting that the former lattice is the canonical one implicitly considered by chung_strong_1951 and which can be found in \ref{['corollary:chung-slln', 'corollary:game-theoretic sllns']}. By \ref{['corollary:game-theoretic sllns']} combined with the fact that $X_1$ is not $\mathcal{P}^\star$-uniformly integrable, we have that $(E_n^\star)_{n \in \mathbb N}$ cannot diverge tail-uniformly on the former lattice. However, it is easy to check that it does on the latter. Nevertheless, once combined with the fact that tail-uniform divergence to $\infty$ implies pointwise divergence to $\infty$, \ref{['corollary:game-theoretic sllns']} yields an extension of ruf2022composite to $q^{\text{th}}$ moments for $q \in (1,2)$ and at the Marcinkiewicz-Zygmund rate of $o(n^{1/q-1})$. Similarly, \ref{['corollary:game-theoretic lils']} yields a composite LIL for random variables with finite second moments. We state these results here for the sake of completeness. Let $\mathcal{P} \subset \overline \mathcal{P}$ and $q \in [1,2)$. If ${\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{U}_\mathsf{P}^{(q)}(x) \to 0$ as ${x \nearrow \infty}$, then $\nu_\mathcal{P}[A_{q\mathrm{\text{-}div}}] = 0$. Furthermore, if ${\sup_{\mathsf{P} \in \mathcal{P}} \,} \mathsf{U}_\mathsf{P}^{(2)}(x) \to 0$ as ${x \nearrow \infty}$, then $\nu_\mathcal{P}[A_{\mathrm{fluc}}] = 0$. The proof of \ref{['theorem:lq-concentration']} proceeds as follows. We begin by decomposing the partial sum $\sum_{i=1}^k X_i$ into three parts. The first is a partial sum consisting of upper-truncated versions of the $X_i$'s where the common truncation level scales with $\varepsilon m^{\lambda / q}$ for some $\lambda \in (0,1 - q/2)$. Note that we will later set $\lambda$ to $1/2 - q/4$ to arrive at the statement of \ref{['theorem:lq-concentration']}. The second consists of a similar partial sum but upper- and lower-truncated to the intervals $(\varepsilon m^{\lambda / q}, (i-1)^{1/q})$ --- where the interval is taken to be the empty set if $\varepsilon m^{\lambda / q} > (i-1)^{1/q}$ --- in particular noting that these intervals now depend on the indices $i \in \{m, m+1, \dots \}$. The third is a partial sum consisting of the remaining parts of the $X_i$'s after being truncated in the first two terms. Recall that our goal is to control $S_k$ time-uniformly with high probability, and we do so for the first term by exploiting the fact that the applied truncation induces sub-Gaussianity of the partial sums. The second and third terms are controlled by exploiting properties of the truncated random variables combined with Kolmogorov's inequality and summation by parts. Of particular note is that we will not simply apply Kronecker's lemma on a set of probability one (a common technique in classical proofs of SLLNs), but our use of summation by parts implicitly plays an analogous role in a non-asymptotic manner. Fix $m \in \mathbb N$, $\varepsilon > 0$, and $\lambda \in (0,1-q/2)$. We begin by considering the following three-term decomposition of $S_k$ and applying the triangle inequality to obtain $\frac{1}{k^{1/q}} \left \lvert S_k \right \rvert \leq \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Y_i \right \rvert + \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Z_i \right \rvert + \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k R_i \right \rvert,$ where Y_i:= X_i \mathds{1} \{ |X_i| \leq \varepsilon m^{\lambda/q} \} - \mathsf E_\mathsf{P} \left [ X_i \mathds{1} \{ |X_i| \leq \varepsilon m^{\lambda/q} \} \right ],Z_i:= X_i \mathds{1} \{ \varepsilon m^{\lambda/q} < |X_i| \leq (i-1)^{1/q} \} - \mathsf E_\mathsf{P} \left [ X_i \mathds{1} \{ \varepsilon m^{\lambda/q} < |X_i| \leq (i-1)^{1/q} \} \right ],R_i:= X_i \mathds{1} \{ |X_i| > \varepsilon m^{\lambda/q} \lor (i-1)^{1/q} \} - \mathsf E_\mathsf{P} \left [ X_i \mathds{1} \{ |X_i| > \varepsilon m^{\lambda/q} \lor (i-1)^{1/q} \} \right ]. In the following three steps, we will derive time-uniform concentration inequalities for the partial sums of $(X_n)_{n \in \mathbb N}$, $(Y_n)_{n \in \mathbb N}$, and $(Z_n)_{n \in \mathbb N}$, respectively, ultimately combining them in the fourth step to obtain the desired result. Observe that by truncating at $\varepsilon m^{\lambda/q}$, the random variable $Y_i$ is supported on a finite interval $[a,b]$ with $b-a = 2 \varepsilon m^{\lambda /q}$ and hence $Y_i$ is sub-Gaussian with variance proxy $(b-a)^2 / 4 = \varepsilon^2 m^{2\lambda /q}$ for each $i \in \mathbb N$. Moreover, these summands are all independent of each other. Exploiting this sub-Gaussianity, we employ a concentration inequality due to howard2018uniform, which implies that for any $\alpha \in (0, 1)$, and any function $h : [0, \infty) \to (0, \infty)$, it holds that $\mathsf{P} \left [ \exists k \in \mathbb N : \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq \varepsilon m^{\lambda/q} \sqrt{c^2 k \left ( \log h \left ( \log(k) \right ) + \log(1/\alpha) \right )} \right ] \leq \alpha \sum_{j=0}^\infty \frac{1}{h(j)},$ where $c := \left ( e^{1/4} + e^{-1/4} \right ) / \sqrt{2}$. Now, define $h(x) := \exp \left ( \left ( e^x + m \right )^{2(1-\lambda) /q - 1} \right ); \qquad x \geq 0.$ Take $\alpha := \exp ( -m^{2(1-\lambda)/q - 1} )$ so that we can re-write \ref{['eq:howard-stitched']} as $\mathsf{P} \left [ \exists k \in \mathbb N : \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq b_k \right ] \leq \exp \left (-m^{2(1-\lambda)/q - 1} \right ) \sum_{j=0}^\infty \frac{1}{h(j)} \leq H \exp \left (-m^{2(1-\lambda)/q - 1} \right ),$ where $H := \sum_{j=0}^\infty \exp ( -(e^j)^{2(1-\lambda)/q - 1} ) \geq \sum_{j=0}^\infty 1/h(j)$ and the boundary $b_k$ is given by $b_k := \varepsilon m^{\lambda/q} \sqrt{c^2 k \left ( (k + m)^{2(1-\lambda) /q-1} + m^{2(1-\lambda)/q - 1} \right )},$ Notice we have for any $k \geq m$, b_k \leq \varepsilon m^{\lambda/q}\sqrt{2 c^2k (k+m)^{2(1-\lambda)/q-1}} \leq \varepsilon \sqrt{2} c m^{\lambda/q} (k+m)^{(1-\lambda)/q} \leq \varepsilon \sqrt{2}c (k+m)^{1/q}. Therefore, we have the following time-uniform bound on ${\sup_{k \geq m}} |k^{-1/q} \sum_{i=1}^k Y_i|$: \mathsf{P} \left [ \sup_{k \geq m} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq 2^{1/q+1/2} c \varepsilon \right ]= \mathsf{P} \left [ \sup_{k \geq m} \frac{1}{(2k)^{1/q}} \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq \sqrt{2} c \varepsilon \right ]\leq \mathsf{P} \left [ \exists k \geq 1 : \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq \sqrt{2}c \varepsilon (k+m)^{1/q} \right ]\leq H \exp \left (-m^{2(1-\lambda)/q - 1} \right ). Notice that $H$ can be upper bounded as follows: H = \sum_{j=0}^\infty \exp \left (- e^{ j(2(1-\lambda)/ q - 1) } \right ) \leq 1 + \int_{0}^\infty \exp \left ( - e^{ y(2(1-\lambda)/ q - 1) } \right )\mathrm{d} y = 1 + \frac{\int_{1}^\infty t^{-1} e^{-t} \mathrm{d} t}{2(1-\lambda)/q - 1}. Note that $\int_1^\infty t^{-1} e^{-t} \mathrm{d} t \leq e^{-1} < 1/2$. Observing that $2^{1/q + 1/2} c \leq 2(e^{1/4} + e^{-1/4}) < 4.13$, we can write the time-uniform bound on ${\sup_{k \geq m}} |k^{-1/q} \sum_{i=1}^k Y_i|$ as \mathsf{P} \left [ \sup_{k \geq m} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Y_i \right \rvert \geq 4.13 \varepsilon \right ]\leq \left ( 1 + \frac{q}{4(1-\lambda) - 2q} \right ) \exp \left(-m^{2(1-\lambda)/q - 1} \right ), which completes Step I. For this step, we will use the following lemma. Let $(a_n)_{n \in \mathbb N}$ be a monotonically nondecreasing and strictly positive sequence and let $(b_n)_{n \in \mathbb N}$ be any real sequence. Then for any $M \in \mathbb N$, we have $\max_{ 1\leq k \leq M} \frac{|\sum_{i=1}^k b_i |}{a_k} \leq 2 \max_{1 \leq k \leq M} \left \lvert \sum_{i=1}^k \frac{b_i}{a_i} \right \rvert.$ For $n \in \mathbb N$, define $b_n' := b_n / a_n$, $T_n := \sum_{i=1}^n b_i, \quad\text{and}\quad T_{n}' := \sum_{i=1}^n b_i'.$ Using summation by parts, we can write $T_k$ for each $k \in \mathbb N$ as T_k = \sum_{i=1}^k a_ib_i' = \sum_{i=1}^k a_i(T_i' - T_{i-1}') = a_k T_k' - \sum_{i=1}^k (a_i - a_{i-1}) T_{i-1}', where we set $a_0 := 0$. Therefore, for each $k \in \mathbb N$, |T_k|\leq a_k |T_k'| + \sum_{i=1}^k (a_i - a_{i-1}) |T_{i-1}'| \leq a_k |T_k'| + \left (\max_{1 \leq j \leq k} |T_{j-1}'| \right )\sum_{i=1}^k (a_i - a_{i-1}) \leq 2 a_k \max_{1 \leq j \leq k} |T_j'|. Dividing both sides by $a_k$ and taking maxima over $k \in \{1, \dots, M\}$ completes the proof. We now continue with Step II. By monotone convergence, \ref{['lemma:maximal weighted sum inequality']}, and Kolmogorov's inequality, we have \mathsf{P} \left [ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Z_i \right \rvert \geq \varepsilon \right ]\leq \lim_{M\nearrow \infty} \mathsf{P} \left [ \max_{1 \leq k \leq M} \left \lvert \sum_{i=1}^k \frac{Z_i}{i^{1/q}} \right \rvert \geq \frac{\varepsilon}{2} \right ] \leq \frac{4}{\varepsilon^2} \mathsf E_\mathsf{P} \left [ \sum_{i=1}^\infty \frac{Z_i^2}{i^{2/q}} \right ]. Since $\mathsf E_\mathsf{P} \left [ Z_i^2 \right ] \leq \mathsf E_\mathsf{P} \left [ X_1^2 \mathds{1} \{ \varepsilon m^{\lambda/q} < |X_1| \leq (i-1)^{1/q} \} \right ]$ for each $i \in \mathbb N$, we get \mathsf E_\mathsf{P} \left [ \sum_{i=1}^\infty \frac{Z_i^2}{i^{2/q}} \right ]= \sum_{i=1}^\infty \mathsf E_\mathsf{P} \left [ \frac{Z_i^2}{i^{2/q}} \right ]= \mathsf E_\mathsf{P} \left [ X_1^2 \mathds{1} \{ |X_1| > \varepsilon m^{\lambda/q} \} \sum_{i=1}^\infty \mathds{1} \{ |X_1|^q \leq i-1 \} \frac{1}{i^{2/q}} \right ]\leq \mathsf E_\mathsf{P} \left [ X_1^2 \mathds{1} \{ |X_1| > \varepsilon m^{\lambda/q} \} \int_{|X_1|^q}^\infty \frac{1}{y^{2/q}}\mathrm{d} y \right ]= \mathsf E_\mathsf{P} \left [ \cancel{X_1^2} \mathds{1} \{ |X_1| > \varepsilon m^{\lambda/q} \} \left ( \frac{2}{q} - 1 \right ) \cancel{X_1^{-2}} |X_1|^q \right ]\leq \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda) Putting the above arguments together, we have the following concentration inequality for ${\sup_{k \geq m}} k^{-1/q} | \sum_{i=1}^k Z_i |$: \mathsf{P} \left [ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k Z_i \right \rvert \geq \varepsilon \right ]\leq \frac{4}{\varepsilon^2}\mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda). For any $i \in \mathbb N$, set $\mu_i := \mathsf E_\mathsf{P} \left [ X_i \mathds{1} \{ |X_i| > \varepsilon m^{\lambda/q} \lor (i-1)^{1/q} \} \right ]$ so that $R_i$ can be written more succinctly as $R_i = X_i \mathds{1} \{ |X_i|^q > \varepsilon^q m^\lambda \lor (i-1) \} - \mu_i.$ Hence we can upper-bound the desired tail probability $\mathsf{P} \left [ {\sup_{k \geq m}} k^{-1/q} | \sum_{i=1}^k R_i | \geq \varepsilon \right ]$ as \mathsf{P} \left [ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k R_i \right \rvert \geq \varepsilon \right ]\leq \underbrace{\mathds{1} \left \{ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k \mu_i \right \rvert \geq \varepsilon \right \}}_{(\star)} + \underbrace{\mathsf{P} \left [ \exists i \in \mathbb{N} : |X_i|^q > \varepsilon^q m^\lambda \lor (i-1) \right ]}_{(\dagger)}. Upper-bounding the indicator $(\star)$, we have (\star)\leq \underbrace{\mathds{1} \left \{ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \mathsf E_\mathsf{P} \left [ X_1 \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} \right ] \geq \frac{\varepsilon}{2} \right \}}_{(\star i)}\quad+ \underbrace{\mathds{1} \left \{ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \sum_{i=2}^k \mathsf E_\mathsf{P} \left [ \frac{(i-1)^{1/q}X_i}{(i-1)^{1/q}} \mathds{1} \{ |X_i|^q > \varepsilon^q m^\lambda \lor (i-1) \} \right ] \geq \frac{\varepsilon}{2} \right \}}_{(\star ii)}. Notice now that since $m \in \mathbb N$, we have that $|X_1| / \varepsilon \leq |X_1|^q / \varepsilon^q$ on the event $\{|X_1|^q > \varepsilon^q m^\lambda\}$ and thus $(\star i)$ can be upper bounded as (\star i) \leq 2\mathsf E_\mathsf{P}\left [\frac{|X_1|}{\varepsilon} \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} \right ] \leq \frac{2}{\varepsilon^q} \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda). Turning to the second term $(\star ii)$, notice that on the event $\{ |X_i|^q > i-1 \}$, we have $|X_i| / (i-1)^{1/q} \leq |X_i|^q / (i-1)$ by virtue of the fact that $q \in [1,2)$, and hence $(\star ii)$ can be upper-bounded as (\star ii)\leq \mathds{1} \left \{ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \sum_{i=2}^k \mathsf E_\mathsf{P} \left [ \frac{(i-1)^{1/q}|X_1|^q}{i-1} \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \lor (i-1) \} \right ] \geq \frac{\varepsilon}{2} \right \}\leq \mathds{1} \left \{ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \sum_{i=2}^k \mathsf E_\mathsf{P} \left [ (i-1)^{1/q-1}|X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} \right ] \geq \frac{\varepsilon}{2}\right \}\leq \mathds{1} \left \{ {\sup_{k \geq m}} \frac{\mathsf E_\mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} \right ]}{k^{1/q}} \int_{1}^{k+1} (y-1)^{1/q-1}\mathrm{d} y \geq \frac{\varepsilon}{2}\right \}= \mathds{1} \left \{ {\sup_{k \geq m}} \frac{\mathsf E_\mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} \right ]}{\cancel{k^{1/q}}} q \cancel{k^{1/q}} \geq \frac{\varepsilon}{2} \right \}\leq \frac{4}{\varepsilon}\mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda). Putting the bounds on $(\star i)$ and $(\star ii)$ together, we have that $(\star) \leq (\star i ) + (\star ii) \leq \left ( \frac{2}{\varepsilon^q} + \frac{4}{\varepsilon} \right ) \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda).$ Turning now to $(\dagger)$, we note that if $|X_i|^q > \varepsilon^q m^\lambda \lor (i-1)$, then $|X_i|^q \mathds{1} \{ |X_i|^q > \varepsilon^q m^\lambda \} > i-1$ and thus we can union bound to obtain (\dagger)\leq \sum_{i=1}^\infty \mathsf{P} \left [ |X_1|^q > \varepsilon^q m^\lambda \lor (i-1) \right ] \leq \mathsf{P} \left [ |X_1|^q > \varepsilon^q m^\lambda \right ] + \sum_{i=2}^\infty \mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} > i-1 \right ]. Now, clearly $\mathsf{P} [|X_1|^q > \varepsilon^q m^\lambda] \leq \mathsf E_\mathsf{P}[|X_1|^q / (\varepsilon^q m^\lambda) \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \}] \leq \varepsilon^{-q}\mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda),$ so we can continue the above upper bound as (\dagger)\leq \frac{1}{\varepsilon^{q}}\mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda)+ \sum_{i=1}^\infty \mathsf{P} \left [ |X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} > i \right ]\leq \frac{1}{\varepsilon^q} \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda) + \int_0^\infty \mathsf{P} [|X_1|^q \mathds{1} \{ |X_1|^q > \varepsilon^q m^\lambda \} > u]\mathrm{d} u= \left ( 1 + \frac{1}{\varepsilon^q} \right )\mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda). Putting the bounds on $(\star)$ and $(\dagger)$ together, we obtain that \mathsf{P} \left [ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k R_i \right \rvert \geq \varepsilon \right ]\leq \left ( 1 + \frac{4}{\varepsilon} + \frac{3}{\varepsilon^q} \right ) \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda) \leq \frac{8}{\varepsilon^2 \land 1} \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda). Finally, combining the results from the previous three steps yields \mathsf{P} \left [ {\sup_{k \geq m}} \frac{1}{k^{1/q}} \left \lvert \sum_{i=1}^k X_i \right \rvert \geq 6.13\varepsilon \right ] \leq c_{q,\lambda} \exp \left ( -m^{2(1-\lambda)/q - 1} \right )+ \frac{12}{\varepsilon^2 \land 1} \mathsf{U}_\mathsf{P}^{(q)}(\varepsilon^q m^\lambda), where $c_{q, \lambda} := (4(1-\lambda) - q)/(4(1-\lambda) - 2q)$. Taking $\lambda = 1/2 - q/4$ completes the proof. Fix $\varepsilon > 0$. Let $V_n := \sum_{i=1}^n \left ( X_i^2 + 2{\widebar \sigma^2} \right )/3$ and $\psi$ be the function given by $\psi(\lambda) := \lambda^2 / 2$, $\lambda \in \mathbb R$. By howard_exponential_2018, we have that $(X_n)_{n \in \mathbb N}$ is a sub-Gaussian process with cumulative variance proxy $(V_n)_{n \in \mathbb N}$, meaning that the exponential process $\exp \left ( \lambda S_n - \psi(\lambda) V_n \right )$ is upper-bounded by a nonnegative supermartingale starting at one with respect to the filtration $(\mathcal{F}_n)_{n\in \mathbb N_0}$ generated by the process $(X_n)_{n \in \mathbb N}$. Therefore, by howard2018uniform, for $\eta = 1+\varepsilon$ and any $v_0 > {\widebar \sigma^2}$ (to be set later), $\mathsf{P} \left [ \exists n \in \mathbb N : V_n \geq v_0 ~\text{and}~ S_n \geq b(V_n) \right ] \leq \sum_{j=\lfloor \log_\eta (v_0 / {\widebar \sigma^2}) \rfloor}^\infty \frac{1}{h(j)},$ where $h(j) := (j+1)^\eta \zeta(\eta)$ so that $\sum_{j=0}^\infty 1/h(j) = 1$ by construction, and where $b(v) := \frac{\eta^{1/4} + \eta^{-1/4}}{\sqrt{2}} \sqrt{v \left ( \eta \log \left ( \log_\eta \left ( \eta v / {\widebar \sigma^2} \right ) \right ) + \log \left ( 2\zeta(\eta) \right )\right )}, \qquad v > 0.$ Note that whenever $v \geq {\widebar \sigma^2}$, the boundary $b(v)$ can be upper-bounded as b(v)= \xi \sqrt{v \left ( \eta\log \log \left ( \eta v / {\widebar \sigma^2} \right ) + \ell_\varepsilon \right )} < \xi \sqrt{v \eta \left ( \log \log \left ( \eta v/{\widebar \sigma^2} \right ) + \ell_\varepsilon \right )}, where we write $\xi := (\eta^{1/4} + \eta^{-1/4}) / \sqrt{2}$ and $\ell_\varepsilon := \log(2 \zeta(\eta) / ( \log(\eta)))$ for succinctness. Putting the above upper bound together with \ref{['eq:invoking-howard-theorem1']}, we have for any $v_0 > {\widebar \sigma^2}$, \mathsf{P} \left [ \exists n \in \mathbb N : V_n \geq v_0 ~\text{and}~ S_n \geq \xi \sqrt{V_n \eta \left ( \log \left ( \log \left ( \eta V_n / {\widebar \sigma^2} \right ) \right ) + \ell_\varepsilon \right )} \right ] \leq \sum_{j=\lfloor \log_\eta (v_0 / {\widebar \sigma^2}) \rfloor}^\infty \frac{1}{h(j)}. Consider next the event $A_m$ for each $m \in \mathbb N$ given by $A_m := \left \{ \forall k \geq m,~ \left \lvert \frac{1}{k} \sum_{i=1}^k X_i^2 - \sigma_\mathsf{P}^2 \right \rvert < \varepsilon {\widebar \sigma^2} \right \}.$ Notice that by definition of $V_k$, we have, on the event $A_m$, for every $k \geq m$, $V_k = \frac{1}{3}\sum_{i=1}^k (X_i^2 + 2 {\widebar \sigma^2}) < \frac{1}{3} \left ( k (\sigma_\mathsf{P}^2 + \varepsilon {\widebar \sigma^2}) + 2k({\widebar \sigma^2} + \varepsilon {\widebar \sigma^2}) \right ) \leq k (1+\varepsilon) {\widebar \sigma^2}.$ Moreover, we have, for every $k \in \mathbb N$, $V_k = \frac{1}{3} \sum_{i=1}^k (X_i^2 + 2 {\widebar \sigma^2}) \geq \frac{2k}{3} {\widebar \sigma^2},$ and hence the following sequence of inequalities: \mathsf{P} \left [ {\sup_{k \geq m}} \frac{|S_k / \widebar \sigma|}{ \xi\sqrt{(1+\varepsilon)^2 k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ]=\ \mathsf{P} \left [ \exists k \geq 1 : k {\widebar \sigma^2} \geq m {\widebar \sigma^2} \text{ and } \frac{|S_k/\widebar \sigma|}{\xi \sqrt{(1+\varepsilon) k \eta \left (\log \log ( \eta (1 + \varepsilon) k ) + \ell_\varepsilon \right )}} \geq 1 \right ]\leq\ \mathsf{P} \left [ \exists k \geq 1 : V_k \geq \frac{2}{3} m{\widebar \sigma^2} \text{ and } \frac{ |S_k|}{ \xi \sqrt{V_k \eta \left ( \log \log ( \eta V_k / {\widebar \sigma^2}) + \ell_\varepsilon \right )}} \geq 1 \right ] + \mathsf{P} \left [ {\sup_{k \geq m}} \left \lvert \frac{1}{k} \sum_{i=1}^k X_i^2 - \sigma_\mathsf{P}^2 \right \rvert \geq \varepsilon {\widebar \sigma^2} \right ]. Analyzing the first term, we apply \ref{['eq:invoking-howard-refined']} with $v_0 = 2m{\widebar \sigma^2} / 3$, noticing that since $m \in \mathbb N \setminus \{1\}$, we have $v_0 \geq 4 {\widebar \sigma^2} /3 > {\widebar \sigma^2}$ and hence \mathsf{P} \left [ \exists k \geq 1 : V_k \geq \frac{2}{3} m {\widebar \sigma^2} \text{ and }\frac{ |S_k|}{\xi\sqrt{V_k \eta \left ( \log \log (\eta V_k / {\widebar \sigma^2} ) + \ell_\varepsilon \right )}} \geq 1 \right ]\leq \sum_{j=\lfloor \log_\eta (2m/3) \rfloor}^\infty \frac{1}{h(j)}\leq \int_{\log_\eta(2m/3) - 1}^\infty \frac{1}{(x+1)^\eta \zeta(\eta)} \mathrm{d} x\leq \frac{ \log_\eta^{1-\eta}(2m/3)}{(\eta-1)\zeta(\eta)}. Analyzing the second term, we have by \ref{['theorem:l1-concentration']}, \mathsf{P} \left [ {\sup_{k \geq m}} \left \lvert \frac{1}{k} \sum_{i=1}^k X_i^2 - \sigma_\mathsf{P}^2 \right \rvert \geq \varepsilon {\widebar \sigma^2} \right ] \leq \frac{262}{(\widebar \sigma^4 \varepsilon^2) \wedge 1} \left ( m^{2\lambda - 1} + \mathsf E_\mathsf{P} \left [|X_1^2 - \sigma_\mathsf{P}^2| \mathds{1} \{ |X_1^2 - \sigma_\mathsf{P}^2| \geq m^\lambda \}\right ] \right ). Combining these two estimates and recalling that $\eta = 1+ \varepsilon$, we get \mathsf{P} \left [ {\sup_{k \geq m}} \frac{| S_k/ \widebar \sigma|}{ \xi\sqrt{(1 + \varepsilon)^2 k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ]\leq\ \frac{\log_{1+\varepsilon}^{-\varepsilon}(2m/3)}{\varepsilon \zeta(1+\varepsilon)} + \frac{262}{(\widebar \sigma^4 \varepsilon^2) \wedge 1} \left ( m^{2\lambda - 1} + \mathsf E_\mathsf{P}[|X_1^2 - \sigma_\mathsf{P}^2| \mathds{1} \{ |X_1^2 - \sigma_\mathsf{P}^2| \geq m^\lambda \}] \right ). Letting $c_\varepsilon := (1+\varepsilon)\xi$ yields the desired result and completes the proof of \ref{['theorem:lil']}. Fix $\varepsilon > 0$ and $m \in \mathbb N \setminus \{1\}$. Applying \ref{['theorem:lil']} to the random variables $(X_n/\sigma_\mathsf{P})_{n \in \mathbb N}$ with ${\widebar \sigma^2} = 1$, we have \mathsf{P} \left [ {\sup_{k \geq m}} \frac{ |S_k/ \sigma_\mathsf{P}|}{c_\varepsilon \sqrt{k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ]\leq\ \frac{1}{\varepsilon \log_{1+\varepsilon}^{\varepsilon}(2m/3) \zeta(1+\varepsilon)} + \frac{262}{\varepsilon^2 \wedge 1} \left ( m^{2\lambda - 1} + \mathsf E_\mathsf{P}[|(X_1/ \sigma_\mathsf{P})^2 - 1| \mathds{1} \{ |(X_1/ \sigma_\mathsf{P})^2 - 1| \geq m^\lambda \}] \right )\leq\ \frac{1}{\varepsilon \log_{1+\varepsilon}^{\varepsilon}(2m/3) \zeta(1+\varepsilon)} + \frac{262}{\varepsilon^2 \wedge 1} \left (m^{2\lambda - 1} + \mathsf{U}_\mathsf{P}^{(2)}(m^\lambda)\right ), where the last inequality uses that $m^\lambda > 1$. Noticing that for each $n \geq m$ we can write the difference $\widehat{\sigma}_n^2 - \sigma_\mathsf{P}^2$ as \widehat{\sigma}_n^2 - \sigma_\mathsf{P}^2 = \frac{1}{n} \sum_{i=1}^n (X_i^2 - \widehat{\mu}_n^2) - \mathsf E_\mathsf{P}[X_1^2] = \frac{1}{n} \sum_{i=1}^n X_i^2 - \mathsf E_\mathsf{P}[X_1^2] - \widehat{\mu}_n^2, we have that on the event $A := \bigcap_{k=m}^\infty \left \{ \left \lvert \frac{1}{k} \sum_{i=1}^k \frac{X_i^2}{\sigma_\mathsf{P}^2} - 1\right \rvert < \varepsilon \text{ and } \left \lvert \frac{1}{k} \sum_{i=1}^k \frac{X_i}{\sigma_\mathsf{P}} \right \rvert < \sqrt{\varepsilon} \right \},$ it holds that $|\widehat{\sigma}_\mathsf{P}^2/\widehat{\sigma}_k^2 - 1| < 2\varepsilon$ and hence $|\sigma_\mathsf{P} / \widehat{\sigma}_k| < \sqrt{1 + 2\varepsilon}$ for all $k \geq m$. Applying \ref{['theorem:l1-concentration']} to $(X_n/\sigma_\mathsf{P})_{n \in \mathbb N}$ and $(X_n^2 / \sigma_\mathsf{P}^2)_{n \in \mathbb N}$, respectively, we have \mathsf{P} \left [ {\sup_{k \geq m}} \left \lvert \frac{1}{k}\sum_{i=1}^k \frac{X_i}{\sigma_\mathsf{P}} \right \rvert \geq \sqrt{\varepsilon} \right ]\leq \frac{262}{\varepsilon \land 1} \left(m^{2\lambda - 1} + \mathsf E_\mathsf{P}\left[\left|\frac{X_1}{ \sigma_\mathsf{P}}\right| \mathds{1} \{ |X_1 / \sigma_\mathsf{P}| \geq m^\lambda \}\right]\right) \leq \frac{262}{\varepsilon^2 \land 1} (m^{2\lambda - 1} + \mathsf{U}_\mathsf{P}^{(2)}(m^\lambda)) and \mathsf{P} \left [ {\sup_{k \geq m}} \left \lvert \frac{1}{k} \sum_{i=1}^k \frac{X_i^2}{\sigma_\mathsf{P}^2} - 1 \right \rvert \geq \varepsilon \right ]\leq \frac{262}{\varepsilon^2 \land 1} \left(m^{2\lambda - 1} + \mathsf E_\mathsf{P}\left[\left|\frac{X_1^2}{ \sigma_\mathsf{P}^2}-1\right| \mathds{1} \{ |X_1^2 / \sigma_\mathsf{P}^2 - 1|\geq m^\lambda \}\right]\right)\leq \frac{262}{\varepsilon^2 \land 1} (m^{2\lambda - 1} + \mathsf{U}_\mathsf{P}^{(2)}(m^\lambda)). Putting all of the previous inequalities together and recalling that on the event $A$, it holds $|\sigma_\mathsf{P} / \widehat{\sigma}_k| < \sqrt{1+2\varepsilon}$, we have \mathsf{P} \left [ {\sup_{k \geq m}} \frac{| S_k / \widehat{\sigma}_k|}{c_\varepsilon \sqrt{(1 + 2\varepsilon) k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ]=\mathsf{P} \left [ {\sup_{k \geq m}} \frac{|\sigma_\mathsf{P} / \widehat{\sigma}_k|}{\sqrt{1+2\varepsilon}} \frac{ |S_k / \sigma_\mathsf{P}|}{c_\varepsilon \sqrt{k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ]\leq \mathsf{P} \left [ {\sup_{k \geq m}} \frac{ |S_k / \sigma_\mathsf{P}|}{c_\varepsilon \sqrt{ k \left ( \log \log ( (1 + \varepsilon )^2 k) + \ell_\varepsilon \right )}} \geq 1 \right ] + \mathsf{P} [A^c]\leq \frac{1}{\varepsilon \log^{\varepsilon}_{1+\varepsilon}(2m/3) \zeta(1+\varepsilon)} + \frac{786}{\varepsilon^2 \land 1} \left ( m^{2\lambda - 1} + \mathsf{U}_\mathsf{P}^{(2)} (m^\lambda) \right ), completing the proof of \ref{['corollary:self-normalized-lil']}. The authors thank Siva Balakrishnan, Raf Frongillo, and Aaditya Ramdas for helpful discussions. Morgane Austern and Lester Mackey. Efficient concentration with Gaussian approximation. arXiv preprint arXiv:2208.09922, 2022.Akshay Balsubramani. Sharp finite-time iterated-logarithm martingale concentration. arXiv preprint arXiv:1405.2639, 2014.Leonard E Baum and Melvin Katz. Convergence rates in the law of large numbers. Transactions of the American Mathematical Society, 120 (1): 108--123, 1965.Istvan Berkes, Ioannis Karatzas, and Walter Schachermayer. Hereditary Hsu-Robbins-Erdös law of large numbers. arXiv preprint arXiv:2503.19484, 2025.Andrew C Berry. The accuracy of the Gaussian approximation to the sum of independent variates. Transactions of the American Mathematical Society, 49 (1): 122--136, 1941.Kai Lai Chung. The strong law of large numbers. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, volume 2, pages 341--353. University of California Press, January 1951.Donald A Darling and Herbert Robbins. Iterated logarithm inequalities. Proceedings of the National Academy of Sciences, 57 (5): 1188--1192, 1967.Donald A Darling and Herbert Robbins. Some further remarks on inequalities for sample sums. Proceedings of the National Academy of Sciences, 60 (4): 1175--1182, 1968.Carl-Gustav Esseen. A moment inequality with an application to the central limit theorem. Scandinavian Actuarial Journal, 1956 (2): 160--170, 1956.Allan Gut. Marcinkiewicz laws and convergence rates in the law of large numbers for random variables with multidimensional indices. The Annals of Probability, pages 469--482, 1978.Allan Gut. Convergence rates for probabilities of moderate deviations for sums of random variables with multidimensional indices. The Annals of Probability, pages 298--313, 1980.Allan Gut and Ulrich Stadtmüller. An intermediate Baum--Katz theorem. Statistics & Probability Letters, 81 (10): 1486--1492, 2011.Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform Chernoff bounds via nonnegative supermartingales. Probability Surveys, 17: 257--317, 2020.Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 2021.Kevin Jamieson and Robert Nowak. Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting. In 48th Annual Conference on Information Sciences and Systems (CISS), pages 1--6. IEEE, 2014.Ioannis Karatzas and Walter Schachermayer. A strong law of large numbers for positive random variables. Illinois Journal of Mathematics, 67 (3): 517--528, 2023a.Ioannis Karatzas and Walter Schachermayer. A weak law of large numbers for dependent random variables. Theory of Probability & Its Applications, 68 (3): 501--509, 2023b.Melvin L Katz. Note on the Berry-Esseen theorem. The Annals of Mathematical Statistics, 34 (3): 1107--1108, 1963.Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On the complexity of best-arm identification in multi-armed bandit models. The Journal of Machine Learning Research, 17 (1): 1--42, 2016.Andrei Nikolaevitch Kolmogorov. Über das gesetz der iterierten logarithmus. Mathematische Annalen, 101: 126--135, 1929.Andrei Nikolaevitch Kolmogorov. Sur la loi forte des grands nombres. Comptes rendus de l'Académie des Sciences, 191: 910--912, 1930.Hartmut Lanzinger and Ulrich Stadtmüller. Baum--Katz laws for certain weighted sums of independent and identically distributed random variables. Bernoulli, 9 (6): 985--1002, 2003.Hartmut Lanzinger and Ulrich Stadtmüller. Refined Baum--Katz laws for weighted sums of iid random variables. Statistics & Probability Letters, 69 (3): 357--368, 2004.Jozef Marcinkiewicz and Antoni Zygmund. Sur les fonctions indépendantes. Fundamenta Mathematicae, 29: 60--90, 1937.Morenikeji Neri. Quantitative strong laws of large numbers. Electronic Journal of Probability, 30: 1--22, 2025.Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Sequential complexities and uniform martingale laws of large numbers. Probability Theory and Related Fields, 161 (1): 111--153, 2015.Johannes Ruf, Martin Larsson, Wouter M Koolen, and Aaditya Ramdas. A composite generalization of Ville's martingale theorem. Electronic Journal of Probability, 2023.Takeyuki Sasai, Kenshi Miyabe, and Akimichi Takemura. Erdos-Feller-Kolmogorov-Petrowsky law of the iterated logarithm for self-normalized martingales: A game-theoretic approach. The Annals of Probability, 47 (2): 1136--1161, 2019.Glenn Shafer and Vladimir Vovk. Probability and Finance: It's Only a Game! John Wiley & Sons, 2005.Glenn Shafer and Vladimir Vovk. Game-Theoretic Foundations for Probability and Finance. John Wiley & Sons, 2019.Joel A Tropp. Probability theory & computational mathematics. CMS/ACM 117, 2023.Jean Ville. Étude critique de la notion de collectif. Bull. Amer. Math. Soc, 45 (11): 824, 1939.Vladimir Vovk. A law of large numbers for predicting several steps ahead. arXiv preprint arXiv:2508.17507, 2025.Ian Waudby-Smith, Martin Larsson, and Aaditya Ramdas. Distribution-uniform strong laws of large numbers. arXiv preprint, 2024.Ian Waudby-Smith, Martin Larsson, and Aaditya Ramdas. Nonasymptotic and distribution-uniform Komlós-Major-Tusnády approximation. arXiv preprint arXiv:2502.06188, 2025.Haoyu Ye and Morgane Austern. Computable bounds for strong approximations with applications. arXiv preprint arXiv:2508.03833, 2025.Shengjia Zhao, Enze Zhou, Ashish Sabharwal, and Stefano Ermon. Adaptive concentration inequalities for sequential decision problems. Advances in Neural Information Processing Systems, 29, 2016.