RLS Framework with Segmentation of the Forgetting Profile and Low Rank Updates

TL;DR

The paper addresses the slow adaptation and numerical conditioning issues of classic Recursive Least Squares with infinite memory by introducing a segmented forgetting profile within a finite moving window. The approach combines low rank updates with a generalized matrix inversion lemma to perform efficient updates that balance rapid response and conditioning, controlled by parameters $\beta$, $\lambda$, $m$, and $p$. The authors demonstrate improved estimation accuracy and stability on low resolution daily temperature data, including better first harmonic estimation and a 30‑day forecast with tight confidence intervals. This framework enables incorporation of prior signal characteristics into RLS estimates, offering a practical tool for robust online system identification and time series forecasting in nonstationary environments.

Abstract

This report describes a new regularization approach based on segmentation of the forgetting profile in sliding window least squares estimation. Each segment is designed to enforce specific desirable properties of the estimator such as rapidity, desired condition number of the information matrix, accuracy, numerical stability, etc. The forgetting profile is divided in three segments, where the speed of estimation is ensured by the first segment, which employs rapid exponential forgetting of recent data.The second segment features a decline in the profile and marks the transition to the third segment, characterized by slow exponential forgetting to reduce the condition number of the information matrix using more distant data. Condition number reduction mitigates error propagation, thereby enhancing accuracy and stability. This approach facilitates the incorporation of a priori information regarding signal characteristics (i.e., the expected behavior of the signal) into the estimator. Recursive and computationally efficient algorithm with low rank updates based on new matrix inversion lemma for moving window associated with this regularization approach is developed. New algorithms significantly improve the approximation accuracy of low resolution daily temperature measurements obtained at the Stockholm Old Astronomical Observatory, thereby enhancing the reliability of temperature predictions.

Figures (4)

• Figure 1: The Figure shows the forgetting profile in the moving window of the size $w$ segmented by red, green and blue lines (RGB lines). The rapidity of estimation is guaranteed by rapid exponential forgetting with small forgetting factor $\beta = 0.92$ associated with the red line (red segment). Such rapid forgetting over the whole window implies ill-conditioning of the information matrix, see red dashed line. Reduction of the condition number is associated with the blue line and the forgetting factor $\lambda = 0.96$. The resulting profile is segmented by the red segment, drop (with the magnitude determined by a positive integer $m$) which is plotted with the green line and blue segment with larger forgetting factor. Past data (associated with blue segment) forms the basis for reduction of the condition number of the information matrix and the variances of parameter estimates, while rapidity is achieved by fast forgetting of recent measurements.
• Figure 2: Comparison of the approximation performance of segmented and exponential forgetting profiles in moving window of the size $w=400$ is shown in this Figure. Daily temperature measurements are plotted with the blue line. The output of the RLS algorithm with rank two updates and $\lambda = 0.99$ is plotted with the black line. The output of RLS algorithm with segmented profile and rank four updates, designed for $p=1$, $\beta = 0.89$, $\lambda = 0.99$, $m=250$ (see Figure \ref{['figseg1']}) is plotted with the red line. Histograms of approximation errors are presented in Figure \ref{['figm1']}. Estimates of the first harmonics are plotted with magenta and green lines for exponential and segmented profiles respectively.
• Figure 3: Histogram of the approximation error for the exponential forgetting profile is plotted with the blue color, and the histogram of the approximation error for the segmented forgetting profile is plotted with the red color. Approximation performance is significantly improved via segmentation of the profile.
• Figure 4: The Figure shows the $30$ days temperature forecast based on prediction of the first harmonic and three sigma confidence interval with estimation of the variance in moving window. It is shown that the seasonal trend can be very well predicted using estimate of the first harmonic of low resolution daily temperature measurements. The prediction estimates the mean, maximum, and minimum temperature values projected 30 days ahead. The accuracy of the prediction is assessed by checking if the actual temperature measurements lie within the confidence intervals established around the predicted first harmonic values. Most observed temperature measurements fall within the confidence intervals of the first harmonic predictions, confirming their reliability.