Braced Fourier Continuation and Regression for Anomaly Detection

TL;DR

The paper addresses efficient nonlinear regression and anomaly detection in one-dimensional data by mitigating Gibbs artifacts through Braced Fourier Continuation. It introduces BFCR, which extends data with bracing points, applies a low-pass filter in the Fourier domain, and reconstructs a trend line while avoiding end-point explosions. The work outlines the BFCR algorithm, its key properties, and two anomaly-detection strategies (internal and edge), along with practical mitigation techniques for volatility changes, pre-existing outliers, and low-noise scenarios. The methodology is implemented in Python with GitHub availability, highlighting potential extensions to higher dimensions and spacing, as well as prospects for reducing computational complexity.

Abstract

In this work, the concept of Braced Fourier Continuation and Regression (BFCR) is introduced. BFCR is a novel and computationally efficient means of finding nonlinear regressions or trend lines in arbitrary one-dimensional data sets. The Braced Fourier Continuation (BFC) and BFCR algorithms are first outlined, followed by a discussion of the properties of BFCR as well as demonstrations of how BFCR trend lines may be used effectively for anomaly detection both within and at the edges of arbitrary one-dimensional data sets. Finally, potential issues which may arise while using BFCR for anomaly detection as well as possible mitigation techniques are outlined and discussed. All source code and example data sets are either referenced or available via GitHub, and all associated code is written entirely in Python.

Figures (9)

• Figure 1: Illustration of the BFCR algorithm on an example data set taken from Wyrick2022, with select portions magnified.
• Figure 2: Illustration of FC on a non-smooth data set taken from FFIEC. Figure 2a depicts the input data, while Figure 2b illustrates the output of the FC process on this data, when FC hyper-parameter $d=12$.
• Figure 3: Illustration of select steps of the BFC algorithm visualized on example data taken from FFIEC. Figure 3a depicts the input data, while Figure 3b depicts the output of steps 2-6. Figure 3c depicts steps 7-9, and Figure 3d depicts the end result of the algorithm.
• Figure 4: Illustration of select steps of the BFCR Algorithm on example data taken from Wyrick2022. Figure 4a depicts step 1 of the algorithm on the example data. Figure 4b depicts steps 3 and part of step 4, namely the FFT modes of the continued data set with zero mean $Y$ both with and without a Sigma Approximation filter. Figure 4c depicts the reconstructed trend from second part of step 4. Figure 4d depicts the end result of the BFCR algorithm on the input data set.
• Figure 5: Illustration of the BFCR internal anomaly detection algorithm on some example data taken from FFIEC. Figure 5a depicts the sample data set. Figure 5b shows the results of step 1 of the algorithm, Figure 5c shows the process and result of step 2 of the algorithm, and Figure 5d shows the end result of the algorithm when assuming a normal distribution in step 3.