Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Dynamic Trend Filtering through Trend Point Detection with Reinforcement Learning

Jihyeon Seong, Sekwang Oh, Jaesik Choi

TL;DR

DTF-net presents an RL-based framework for trend filtering that directly detects Dynamic Trend Points (DTPs) via Trend Point Detection formulated as an MDP, then reconstructs dynamic trends by interpolation. It defines a discrete-action policy trained with a forecasting-$MSE$ reward, augmented by random reward-sampling to mitigate overfitting, enabling per-subsequence smoothness that preserves abrupt changes. Across synthetic and real data, including Nasdaq and non-stationary TSF benchmarks, DTF-net outperforms traditional trend filtering, CPD, and anomaly-detection baselines in capturing abrupt changes and improving forecasting when abrupt changes are informative. The approach demonstrates that integrating abrupt-change cues into forecasting can enhance predictive performance without forcing overall smoothness, with potential extensions to multivariate settings and broader RL-based trend analysis.

Abstract

Trend filtering simplifies complex time series data by applying smoothness to filter out noise while emphasizing proximity to the original data. However, existing trend filtering methods fail to reflect abrupt changes in the trend due to `approximateness,' resulting in constant smoothness. This approximateness uniformly filters out the tail distribution of time series data, characterized by extreme values, including both abrupt changes and noise. In this paper, we propose Trend Point Detection formulated as a Markov Decision Process (MDP), a novel approach to identifying essential points that should be reflected in the trend, departing from approximations. We term these essential points as Dynamic Trend Points (DTPs) and extract trends by interpolating them. To identify DTPs, we utilize Reinforcement Learning (RL) within a discrete action space and a forecasting sum-of-squares loss function as a reward, referred to as the Dynamic Trend Filtering network (DTF-net). DTF-net integrates flexible noise filtering, preserving critical original subsequences while removing noise as required for other subsequences. We demonstrate that DTF-net excels at capturing abrupt changes compared to other trend filtering algorithms and enhances forecasting performance, as abrupt changes are predicted rather than smoothed out.

Towards Dynamic Trend Filtering through Trend Point Detection with Reinforcement Learning

TL;DR

DTF-net presents an RL-based framework for trend filtering that directly detects Dynamic Trend Points (DTPs) via Trend Point Detection formulated as an MDP, then reconstructs dynamic trends by interpolation. It defines a discrete-action policy trained with a forecasting- reward, augmented by random reward-sampling to mitigate overfitting, enabling per-subsequence smoothness that preserves abrupt changes. Across synthetic and real data, including Nasdaq and non-stationary TSF benchmarks, DTF-net outperforms traditional trend filtering, CPD, and anomaly-detection baselines in capturing abrupt changes and improving forecasting when abrupt changes are informative. The approach demonstrates that integrating abrupt-change cues into forecasting can enhance predictive performance without forcing overall smoothness, with potential extensions to multivariate settings and broader RL-based trend analysis.

Abstract

Trend filtering simplifies complex time series data by applying smoothness to filter out noise while emphasizing proximity to the original data. However, existing trend filtering methods fail to reflect abrupt changes in the trend due to `approximateness,' resulting in constant smoothness. This approximateness uniformly filters out the tail distribution of time series data, characterized by extreme values, including both abrupt changes and noise. In this paper, we propose Trend Point Detection formulated as a Markov Decision Process (MDP), a novel approach to identifying essential points that should be reflected in the trend, departing from approximations. We term these essential points as Dynamic Trend Points (DTPs) and extract trends by interpolating them. To identify DTPs, we utilize Reinforcement Learning (RL) within a discrete action space and a forecasting sum-of-squares loss function as a reward, referred to as the Dynamic Trend Filtering network (DTF-net). DTF-net integrates flexible noise filtering, preserving critical original subsequences while removing noise as required for other subsequences. We demonstrate that DTF-net excels at capturing abrupt changes compared to other trend filtering algorithms and enhances forecasting performance, as abrupt changes are predicted rather than smoothed out.
Paper Structure (48 sections, 1 theorem, 17 equations, 11 figures, 8 tables, 1 algorithm)

This paper contains 48 sections, 1 theorem, 17 equations, 11 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Trend Filtering
  4. Extreme Value Theorem
  5. Markov Decision Process and Reinforcement Learning
  6. Dynamic Trend Filtering Network
  7. Trend Point Detection
  8. Environment Definition
  9. Episode and State for DTF-net
  10. Reward Function of DTF-net
  11. GP and Reward Function Learning
  12. Forecasting Reward Function of DTF-net
  13. Experiment
  14. Trend Filtering Analysis
  15. Experimental Settings
  16. ...and 33 more sections

Key Result

Theorem 1

If the distribution in Equation (eq:extreme) is not degenerate to 0 under a linear transformation of $y$, the distribution of the class with the non-degenerate distribution $G(y)$ should be as follows:

Figures (11)

  • Figure 1: Dynamic Trend Filtering. DTF-net extracts dynamic trends from time series data. Dynamic Trend Points (DTPs) are determined based on action predictions, and the dynamic trend is extracted through interpolation. The agent's action prediction directly influences the variation in trend extraction.
  • Figure 2: DTF-net Architecture. DTF-net has three processes to detect DTPs: 1) The agent predicts actions within a discrete space; 2) With the predicted actions, trends are extracted by interpolating them; 3) The agent is updated through the forecasting sum-of-squares function as a reward; with time series data $X$ and trend $\mathcal{T}$ as inputs. For the reward calculation, as demonstrated in (b)-Case 3, when DTF-net successfully identifies abrupt changes, the prediction outcomes significantly improve, resulting in the highest reward.
  • Figure 3: Synthetic Data.
  • Figure 4: Qualitative comparison with $\ell_1$ and DTF-net. The figure illustrates the trends obtained from the $\ell_1$ and DTF-net using the Nasdaq intraday dataset. The red line denotes the output of each trend filtering method, with red vertical boxes indicating arbitrarily set abrupt changes. The blue dots denote the captured abrupt changes, while the sky-blue dots highlight the constant smoothness from $\ell_1$. Notably, DTF-net has the capability to apply varying levels of smoothness to individual sub-sequences.
  • Figure 5: Qualitative analysis of the impact of abrupt changes on TSF. We conduct forecasting experiments to evaluate the influence of trends incorporating extreme values with long-heavy tails on two datasets, ETTh1 and Exchange rate (EXC). The figure illustrates that including abrupt changes (red) in forecasting plays a crucial role without undergoing smoothing (blue). It is evident that the results appear smoother when extreme values are excluded (depicted by the blue line) in both short-term (24 pred) and long-term (336 pred) forecasting.
  • ...and 6 more figures

Theorems & Definitions (1)

  • Theorem 1: Extreme Value Theory fisher1928limitingding2019modeling