Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Detection of Changes in Moment-Based Projections: When to Retrain Deep Learners or Update Portfolios?

Ansgar Steland

TL;DR

The paper tackles online detection of changes in the moment-based projections used to monitor deep learning predictions and to decide when retraining or portfolio updates are needed. It develops open-end and closed-end monitoring frameworks based on cumulative second-moment statistics, provides Gaussian-approximation-based null and local-alternative results, and analyzes projection estimators under nonstationarity and sparsity. Thresholded covariance/precision estimation is developed to handle high-dimensional settings, enabling reliable projection-based monitoring. The methods are demonstrated through deep learning monitoring and financial portfolio applications with simulations showing fast detection, controlled false alarms, and substantial potential reductions in retraining and trading costs.

Abstract

Training deep learning neural networks often requires massive amounts of computational ressources. We propose to sequentially monitor network predictions to trigger retraining only if the predictions are no longer valid. This can reduce drastically computational costs and opens a door to green deep learning. Our approach is based on the relationship to projected second moments monitoring, a problem also arising in other areas such as computational finance. Various open-end as well as closed-end monitoring rules are studied under mild assumptions on the training sample and the observations of the monitoring period. The results allow for high-dimensional non-stationary time series data and thus, especially, non-i.i.d. training data. Asymptotics is based on Gaussian approximations of projected partial sums allowing for an estimated projection vector. Estimation of projection vectors is studied both for classical non-$\ell_0$-sparsity as well as under sparsity. For the case that the optimal projection depends on the unknown covariance matrix, hard- and soft-thresholded estimators are studied. The method is analyzed by simulations and supported by synthetic data experiments.

Online Detection of Changes in Moment-Based Projections: When to Retrain Deep Learners or Update Portfolios?

TL;DR

The paper tackles online detection of changes in the moment-based projections used to monitor deep learning predictions and to decide when retraining or portfolio updates are needed. It develops open-end and closed-end monitoring frameworks based on cumulative second-moment statistics, provides Gaussian-approximation-based null and local-alternative results, and analyzes projection estimators under nonstationarity and sparsity. Thresholded covariance/precision estimation is developed to handle high-dimensional settings, enabling reliable projection-based monitoring. The methods are demonstrated through deep learning monitoring and financial portfolio applications with simulations showing fast detection, controlled false alarms, and substantial potential reductions in retraining and trading costs.

Abstract

Training deep learning neural networks often requires massive amounts of computational ressources. We propose to sequentially monitor network predictions to trigger retraining only if the predictions are no longer valid. This can reduce drastically computational costs and opens a door to green deep learning. Our approach is based on the relationship to projected second moments monitoring, a problem also arising in other areas such as computational finance. Various open-end as well as closed-end monitoring rules are studied under mild assumptions on the training sample and the observations of the monitoring period. The results allow for high-dimensional non-stationary time series data and thus, especially, non-i.i.d. training data. Asymptotics is based on Gaussian approximations of projected partial sums allowing for an estimated projection vector. Estimation of projection vectors is studied both for classical non--sparsity as well as under sparsity. For the case that the optimal projection depends on the unknown covariance matrix, hard- and soft-thresholded estimators are studied. The method is analyzed by simulations and supported by synthetic data experiments.
Paper Structure (31 sections, 16 theorems, 190 equations, 1 figure, 1 table)

This paper contains 31 sections, 16 theorems, 190 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Monitoring Framework
  3. Open-end monitoring
  4. Closed--end monitoring
  5. Application to deep learners
  6. Asymptotics
  7. Assumptions
  8. Basic estimation under nonstationarity: Non--asymptotic guarantees
  9. A sequential Gaussian approximation
  10. Detector asymptotics under $H_0$ and local alternatives
  11. Estimating projections depending on covariance and precision matrices
  12. Applications and Simulations
  13. Deep neural networks
  14. Financial portfolio optimization
  15. Experimental support and simulations
  16. ...and 16 more sections

Key Result

Proposition 1

Suppose that $\mu_{\bm t, \bm n}^{(m)}, \bm t \in \mathbb{N}^k, \bm n \in \mathbb{N}_0^{k d_m }, k \ge 1$, are real numbers such that for all $\bm t \in \mathbb{N}^k$, $\mu_{\bm t, \bm n}^{(m)}, \bm n \in \mathbb{N}_0^{k d_m },$ satisfy the Riesz--Haviland condition of positive semidefiniteness of t Then, for any $1 \le t_1 < \cdots < t_k$, $k \in \mathbb{N}$, there exists a distribution $Q_{t_1,\

Figures (1)

  • Figure 1: Experiments: The top panel shows the synthetic data set. Before the first change data points are black, after the first and before the second one blue, and data after both changes are in red. The left (right) panel depicts the detector statistic $D_{proj,k}$ ($D_{res,k}$) and the boundary $B_k$. The method reacts quickly and there are no false alarms.

Theorems & Definitions (19)

  • Example 1
  • Example 2
  • Proposition 1
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 3
  • Remark 1
  • Theorem 4
  • Theorem 5
  • ...and 9 more