Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

COTN: A Chaotic Oscillatory Transformer Network for Complex Volatile Systems under Extreme Conditions

Boyan Tang, Yilong Zeng, Xuanhao Ren, Peng Xiao, Yuhan Zhao, Raymond Lee, Jianghua Wu

TL;DR

The paper tackles forecasting in chaotic volatile systems such as electricity and finance under extreme conditions by introducing the Chaotic Oscillatory Transformer Network (COTN). COTN fuses Lee Oscillator-based activations with Max-over-Time pooling and a λ-gated fusion to a Transformer, augmented by an Autoencoder Self-Regressive module and architectural enhancements from Distilled Attention Transformer for robustness and efficiency. Empirical results show COTN outperforming strong baselines like Informer by up to 17% and GARCH by up to 40% across multiple datasets, with notable gains in high-volatility regimes. The approach offers a practical, anomaly-resilient forecasting tool for critical-infrastructure and financial markets, with potential applicability to other high-variance domains.

Abstract

Accurate prediction of financial and electricity markets, especially under extreme conditions, remains a significant challenge due to their intrinsic nonlinearity, rapid fluctuations, and chaotic patterns. To address these limitations, we propose the Chaotic Oscillatory Transformer Network (COTN). COTN innovatively combines a Transformer architecture with a novel Lee Oscillator activation function, processed through Max-over-Time pooling and a lambda-gating mechanism. This design is specifically tailored to effectively capture chaotic dynamics and improve responsiveness during periods of heightened volatility, where conventional activation functions (e.g., ReLU, GELU) tend to saturate. Furthermore, COTN incorporates an Autoencoder Self-Regressive (ASR) module to detect and isolate abnormal market patterns, such as sudden price spikes or crashes, thereby preventing corruption of the core prediction process and enhancing robustness. Extensive experiments across electricity spot markets and financial markets demonstrate the practical applicability and resilience of COTN. Our approach outperforms state-of-the-art deep learning models like Informer by up to 17% and traditional statistical methods like GARCH by as much as 40%. These results underscore COTN's effectiveness in navigating real-world market uncertainty and complexity, offering a powerful tool for forecasting highly volatile systems under duress.

COTN: A Chaotic Oscillatory Transformer Network for Complex Volatile Systems under Extreme Conditions

TL;DR

The paper tackles forecasting in chaotic volatile systems such as electricity and finance under extreme conditions by introducing the Chaotic Oscillatory Transformer Network (COTN). COTN fuses Lee Oscillator-based activations with Max-over-Time pooling and a λ-gated fusion to a Transformer, augmented by an Autoencoder Self-Regressive module and architectural enhancements from Distilled Attention Transformer for robustness and efficiency. Empirical results show COTN outperforming strong baselines like Informer by up to 17% and GARCH by up to 40% across multiple datasets, with notable gains in high-volatility regimes. The approach offers a practical, anomaly-resilient forecasting tool for critical-infrastructure and financial markets, with potential applicability to other high-variance domains.

Abstract

Accurate prediction of financial and electricity markets, especially under extreme conditions, remains a significant challenge due to their intrinsic nonlinearity, rapid fluctuations, and chaotic patterns. To address these limitations, we propose the Chaotic Oscillatory Transformer Network (COTN). COTN innovatively combines a Transformer architecture with a novel Lee Oscillator activation function, processed through Max-over-Time pooling and a lambda-gating mechanism. This design is specifically tailored to effectively capture chaotic dynamics and improve responsiveness during periods of heightened volatility, where conventional activation functions (e.g., ReLU, GELU) tend to saturate. Furthermore, COTN incorporates an Autoencoder Self-Regressive (ASR) module to detect and isolate abnormal market patterns, such as sudden price spikes or crashes, thereby preventing corruption of the core prediction process and enhancing robustness. Extensive experiments across electricity spot markets and financial markets demonstrate the practical applicability and resilience of COTN. Our approach outperforms state-of-the-art deep learning models like Informer by up to 17% and traditional statistical methods like GARCH by as much as 40%. These results underscore COTN's effectiveness in navigating real-world market uncertainty and complexity, offering a powerful tool for forecasting highly volatile systems under duress.

Paper Structure

This paper contains 18 sections, 12 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Forecasting Models for Complex Volatile Systems
  4. The Lee Oscillator for Modeling Chaotic Dynamics
  5. Methodology
  6. Data Description and Preprocessing
  7. Data Source and Basic Details
  8. Data Processing
  9. Parameterized Lee Oscillators Activation Units
  10. "Max-over-Time" Pooling
  11. $\lambda$-Gated Lee Activation Module
  12. Architectural Enhancements for Robustness and Efficiency in COTN
  13. Case Study
  14. Characterizing the Dynamic Activation Unit
  15. Efficiency Enhancement via Pretraining with GELU
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Neural architecture of Lee Oscillator.
  • Figure 2: Eight major categories of bifurcations for Lee Oscillator. (Adapted from lee2019chaotic)
  • Figure 3: Neural architecture of LORS
  • Figure 4: Conceptual illustration of a meta activation function derived from a Lee Oscillator via Max-over-Time pooling.
  • Figure 5: MAE and MSE distribution for Lee oscillator type1
  • ...and 1 more figures