LISA: Laplacian In-context Spectral Analysis

TL;DR

LISA addresses forecasting of nonlinear, potentially nonstationary time series by embedding history via Takens delay coordinates and diffusion-map spectral analysis to obtain diffusion coordinates, then decoding with a frozen Gaussian-Process Latent Model. It introduces two lightweight, inference-time in-context adapters—an In-Context Gaussian Process (ICGP) and an In-Context Nadaraya-Watson (ICNW) mechanism—that operate on the latent space using only the observed prefix, avoiding gradient-based retraining. Across stationary chaotic attractors, nonstationary regime-switching dynamics, and real electricity-load data, the approach improves over the frozen baseline and yields better dynamical fidelity, particularly when longer context reveals useful analog information. The method provides a principled link between in-context adaptation and nonparametric spectral methods for dynamical systems, with practical implications for long-horizon forecasting in complex, evolving environments.

Abstract

We propose Laplacian In-context Spectral Analysis (LISA), a method for inference-time adaptation of Laplacian-based time-series models using only an observed prefix. LISA combines delay-coordinate embeddings and Laplacian spectral learning to produce diffusion-coordinate state representations, together with a frozen nonlinear decoder for one-step prediction. We introduce lightweight latent-space residual adapters based on either Gaussian-process regression or an attention-like Markov operator over context windows. Across forecasting and autoregressive rollout experiments, LISA improves over the frozen baseline and is often most beneficial under changing dynamics. This work links in-context adaptation to nonparametric spectral methods for dynamical systems.

Figures (12)

• Figure 1: LISA schematic. The NLSA encoder constructs diffusion coordinates from the context sequence via delay embedding and Laplacian spectral analysis. The in-context mechanism (ICM) performs a lightweight, prompt-specific nonparametric update (Markov-operator or GP-based), producing an adapted model instance used by the GPLM decoder to reconstruct or predict outputs for query samples.
• Figure 2: Rössler: phase-space rollout visualization. Gray shows a subsampled training trajectory (background). Black is a ground-truth test rollout from a fixed start. Colored curves are forecasts from the same start: the NLSA baseline (context $\ell=L$) and the in-context methods (LISA/ALSA with $\ell>L$). In chaotic systems, forecasts may dephase from the truth while remaining close to the attractor geometry; this visualization highlights attractor fidelity beyond strict pointwise tracking.
• Figure 3: Rössler: context-length response curves. Scalar forecast metrics as a function of context length $\ell$ (multiples of the base window length $L$). Each curve reports mean $\pm$ one standard deviation across multiple test starts. Panels show MSE, spectral divergence computed from Welch PSD estimates (JS/KL-style), ACF-MSE, and MMD$^2$ (RFF).
• Figure 4: Regime-switch Lorenz--63: scalar metrics vs context length in regime B. Forecast metrics in the shifted regime B as a function of context length $\ell$, aggregated as mean $\pm 1$ standard deviation over multiple test starts.
• Figure 5: Regime-switch Lorenz--63: scalar metrics vs in-context temperature at fixed long context. Metrics in regime B as a function of a temperature parameter that controls the locality of in-context matching (larger values correspond to smoother/more global mixing).