Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Statistical Estimation of Edge-Reinforced Random Walks

Qinghua, Ding, Venkat Anantharam

TL;DR

The paper tackles the problem of estimating the initial edge weights $A$ of edge-reinforced random walks from trajectory data on finite graphs. It leverages the representation of ERRWs as random walks in random environments via the magic formula, and introduces a generalized method of moments that uses moments of $U_e=P_{ij}P_{ji}$ to recover $A$. A non-asymptotic analysis is developed by exploiting a hyperbolic Gaussian structure, yielding explicit sample-size and trajectory-length bounds through bounds on the random environment (the $\beta$-field and the $\phi$-field) and the resulting cover time. The results show that a single trajectory is insufficient for parameter identification, motivate a two-stage learning framework, and provide concrete upper and lower bounds on the required number of trajectories and their lengths, with extensions to integer-weight cases and a discussion of open directions for future work.

Abstract

Reinforced random walks (RRWs), including vertex-reinforced random walks (VRRWs) and edge-reinforced random walks (ERRWs), model random walks where the transition probabilities evolve based on prior visitation history~\cite{mgr, fmk, tarres, volkov}. These models have found applications in various areas, such as network representation learning~\cite{xzzs}, reinforced PageRank~\cite{gly}, and modeling animal behaviors~\cite{smouse}, among others. However, statistical estimation of the parameters governing RRWs remains underexplored. This work focuses on estimating the initial edge weights of ERRWs using observed trajectory data. Leveraging the connections between an ERRW and a random walk in a random environment (RWRE)~\cite{mr, mr2}, as given by the so-called "magic formula", we propose an estimator based on the generalized method of moments. To analyze the sample complexity of our estimator, we exploit the hyperbolic Gaussian structure embedded in the random environment to bound the fluctuations of the underlying random edge conductances.

On Statistical Estimation of Edge-Reinforced Random Walks

TL;DR

The paper tackles the problem of estimating the initial edge weights of edge-reinforced random walks from trajectory data on finite graphs. It leverages the representation of ERRWs as random walks in random environments via the magic formula, and introduces a generalized method of moments that uses moments of to recover . A non-asymptotic analysis is developed by exploiting a hyperbolic Gaussian structure, yielding explicit sample-size and trajectory-length bounds through bounds on the random environment (the -field and the -field) and the resulting cover time. The results show that a single trajectory is insufficient for parameter identification, motivate a two-stage learning framework, and provide concrete upper and lower bounds on the required number of trajectories and their lengths, with extensions to integer-weight cases and a discussion of open directions for future work.

Abstract

Reinforced random walks (RRWs), including vertex-reinforced random walks (VRRWs) and edge-reinforced random walks (ERRWs), model random walks where the transition probabilities evolve based on prior visitation history~\cite{mgr, fmk, tarres, volkov}. These models have found applications in various areas, such as network representation learning~\cite{xzzs}, reinforced PageRank~\cite{gly}, and modeling animal behaviors~\cite{smouse}, among others. However, statistical estimation of the parameters governing RRWs remains underexplored. This work focuses on estimating the initial edge weights of ERRWs using observed trajectory data. Leveraging the connections between an ERRW and a random walk in a random environment (RWRE)~\cite{mr, mr2}, as given by the so-called "magic formula", we propose an estimator based on the generalized method of moments. To analyze the sample complexity of our estimator, we exploit the hyperbolic Gaussian structure embedded in the random environment to bound the fluctuations of the underlying random edge conductances.

Paper Structure

This paper contains 14 sections, 10 theorems, 169 equations.

Table of Contents

  1. Introduction
  2. ERRW as random walk in a random environment(RWRE)
  3. The estimation problem of interest
  4. Learning the parameters of Edge-Reinforced Random Walks
  5. Insufficiency of a single trajectory
  6. The maximum likelihood estimator
  7. Estimation via a generalized moment method
  8. Estimating the Moments
  9. Non-asymptotic analysis
  10. Hyperbolic Gaussian measure on a tree
  11. Hyperbolic Gaussian measure for a general graph
  12. Bounds on $P_{ij}$, $\pi_*$, and the cover time
  13. A non-asymptotic sample complexity bound
  14. Open Problems and Future Directions

Key Result

Lemma 2.1

Consider an ERRW on a connected graph $G=(V, E)$. Given a vertex $v_0\in V$ with degree $\deg(v_0)\geq 2$ and initial edge weights $A\in\mathbb{R}_{++}^E$, we sample a random environment $W$ according to the mixing measure in eqn:mixing. Denote the random transition matrix induced by the random weig Here $\Gamma(x), \forall x>0$ is the Gamma function, and in the conditions above we assume that $i,

Theorems & Definitions (23)

  • Remark 1: Positive-integer-valued edge weights
  • Lemma 2.1: Moment formulas
  • proof
  • Lemma 2.2: Independence of the gradient field
  • proof
  • Lemma 2.3: Upper bounds on $\sup_{\vv{e} \in \vv{E}} |\nabla\phi(\vv{e})|$ and $\sup_{i\in V}|\phi_i|$
  • proof
  • Remark 2.3.1: Tightness of the bound on $\sup_{i\in V}|\phi_i|$.
  • Lemma 2.4: Generating function bound
  • proof
  • ...and 13 more