Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic approximation in non-markovian environments revisited

Vivek Shripad Borkar

Abstract

Based on some recent work of the author on stochastic approximation in non-markovian environments, the situation when the driving random process is non-ergodic in addition to being non-markovian is considered. Using this, we propose an analytic framework for understanding transformer based learning, specifically, the `attention' mechanism, and continual learning, both of which depend on the entire past in principle.

Stochastic approximation in non-markovian environments revisited

Abstract

Based on some recent work of the author on stochastic approximation in non-markovian environments, the situation when the driving random process is non-ergodic in addition to being non-markovian is considered. Using this, we propose an analytic framework for understanding transformer based learning, specifically, the `attention' mechanism, and continual learning, both of which depend on the entire past in principle.
Paper Structure (4 sections, 1 theorem, 12 equations)

This paper contains 4 sections, 1 theorem, 12 equations.

Table of Contents

  1. Introduction
  2. Stochastic approximation with Markov noise
  3. Non-markovian environment
  4. Non-markovian SA in machine learning

Key Result

Theorem 2.1

Almost surely, $x(n) \to$ an internally chain transitive invariant set of ODE.

Theorems & Definitions (1)

  • Theorem 2.1