Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transfer Learning and Locally Linear Regression for Locally Stationary Time Series

Jinwoo Park

TL;DR

This work tackles nonstationarity in time series by embedding it within a locally stationary nonparametric regression framework and develops a multivariate locally linear estimator that achieves sharp uniform convergence under strong mixing. It then advances transfer learning for nonparametric regression by linking a small target series with a large related source through a smooth bias function, yielding bias-corrected estimators that borrow information across domains. Theoretical results establish improved nonstationary remainder bounds for locally linear smoothing and articulate phase-transition conditions under which cross-domain information improves estimation, supported by simulation and an empirical fuel-price study. Practically, the methods enable principled information sharing across heterogeneous time-varying domains and provide guidance on when transfer learning is expected to help in dependent, nonstationary settings. The combination of LL estimation with transfer learning offers robust, data-efficient nonparametric inference for evolving systems with cross-domain similarity.

Abstract

This paper investigates locally linear regression for locally stationary time series and develops theoretical results for locally linear smoothing and transfer learning. Existing analyses have focused on local constant estimators and given samples, leaving the principles of transferring knowledge from auxiliary sources across heterogeneous time-varying domains insufficiently established. We derive uniform convergence for multivariate locally linear estimators under strong mixing. The resulting error expansion decomposes stochastic variation, smoothing bias, and a term induced by local stationarity. This additional term, originating from the locally stationary structure, has smaller order than in the Nadaraya-Watson benchmark, explaining the improved local linear performance. Building on these results, we propose bias-corrected transfer learned estimators that connect a sparsely observed series with densely observed related sources through a smoothly varying bias function defined over rescaled time and covariates. An additional refinement shows how local temporal adjustment of this bias enhances stability and enables efficient information borrowing across domains. Simulation studies and an empirical analysis of international fuel prices support the theoretical predictions and demonstrate the practical advantages of transfer learning.

Transfer Learning and Locally Linear Regression for Locally Stationary Time Series

TL;DR

This work tackles nonstationarity in time series by embedding it within a locally stationary nonparametric regression framework and develops a multivariate locally linear estimator that achieves sharp uniform convergence under strong mixing. It then advances transfer learning for nonparametric regression by linking a small target series with a large related source through a smooth bias function, yielding bias-corrected estimators that borrow information across domains. Theoretical results establish improved nonstationary remainder bounds for locally linear smoothing and articulate phase-transition conditions under which cross-domain information improves estimation, supported by simulation and an empirical fuel-price study. Practically, the methods enable principled information sharing across heterogeneous time-varying domains and provide guidance on when transfer learning is expected to help in dependent, nonstationary settings. The combination of LL estimation with transfer learning offers robust, data-efficient nonparametric inference for evolving systems with cross-domain similarity.

Abstract

This paper investigates locally linear regression for locally stationary time series and develops theoretical results for locally linear smoothing and transfer learning. Existing analyses have focused on local constant estimators and given samples, leaving the principles of transferring knowledge from auxiliary sources across heterogeneous time-varying domains insufficiently established. We derive uniform convergence for multivariate locally linear estimators under strong mixing. The resulting error expansion decomposes stochastic variation, smoothing bias, and a term induced by local stationarity. This additional term, originating from the locally stationary structure, has smaller order than in the Nadaraya-Watson benchmark, explaining the improved local linear performance. Building on these results, we propose bias-corrected transfer learned estimators that connect a sparsely observed series with densely observed related sources through a smoothly varying bias function defined over rescaled time and covariates. An additional refinement shows how local temporal adjustment of this bias enhances stability and enables efficient information borrowing across domains. Simulation studies and an empirical analysis of international fuel prices support the theoretical predictions and demonstrate the practical advantages of transfer learning.

Paper Structure

This paper contains 40 sections, 5 theorems, 249 equations, 17 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Kernel estimation for locally stationary time series
  3. Local Stationarity
  4. Multivariate Locally Linear Estimation for locally stationary time series
  5. Estimation Procedure
  6. Assumptions
  7. Uniform Convergence rates for kernel averages
  8. Uniform convergence rates for multivariate locally linear estimates
  9. Transfer learning framework
  10. Model Setup
  11. Assumptions for transfer learning
  12. Transfer Learning for NW estimates
  13. Estimation Procedure
  14. Convergence Rate Analysis
  15. Transfer Learning for Multivariate LL estimates
  16. ...and 25 more sections

Key Result

Theorem 3.1

Assume that (K1)–(K3) are satisfied with and that the kernel $K$ fulfills (C5). Moreover, assume the bandwidth $h$ satisfy Finally, let $S$ be a compact subset of $\mathbb{R}^d$ that satisfies CS. Then it holds that where $\|\cdot\|_{2}$ is euclidean norm defined in $\mathbb{R}^{d+2}$.

Figures (17)

  • Figure 1: Mean of median grid errors versus $\gamma$ for the quadratic bias family. Each curve represents the average over fifty replications for the Nadaraya–Watson and locally linear estimators trained on the target data and their transfer-learning counterparts.
  • Figure 2: Mean of median grid errors versus $\gamma$ for the cubic bias family. The curves illustrate the bias–variance transition predicted by theory: transfer-learning estimators achieve lower error for small $|\gamma|$ but converge to the target-only estimators as the discrepancy curvature increases.
  • Figure 3: Mean of median grid errors versus $\gamma$ for the exponential bias family. The transfer-learning estimators display the same "V"-shaped pattern observed in the polynomial cases, and the locally linear transfer-learning estimator remains superior across the entire range of $\gamma$.
  • Figure 4: Boxplots of median grid errors for the quadratic bias family across fifty replications. Each box corresponds to one value of $\gamma$ and compares the estimators fitted on the target sample with their transfer-learning counterparts.
  • Figure 5: Boxplots of median grid errors for the cubic bias family across fifty replications. The results confirm that the transfer-learning estimators provide substantial improvement near $\gamma=0$ and gradually converge to the target-only estimators as curvature increases.
  • ...and 12 more figures

Theorems & Definitions (10)

  • Definition 2.1: Local Stationarity; cf. Vogt_2012
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 4.1
  • Theorem 4.2
  • proof
  • proof
  • Lemma A.1: Theorem 2.1 in Liebscher1996
  • proof
  • proof