Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Time Encoding via Learnable Transformation Functions

Xi Chen, Yateng Tang, Jiarong Xu, Jiawei Zhang, Siwei Zhang, Sijia Peng, Xuehao Zheng, Yun Xiong

TL;DR

This work addresses the limitation of fixed-pattern time encodings in real-world temporal data by introducing Learnable Transformation-based Generalized Time Encoding (LeTE). LeTE parameterizes nonlinear time transformations with learnable Fourier series or B-spline components, and combines them into a flexible, dimension-efficient embedding that generalizes prior encodings while remaining invariant to time rescaling. Across event-based vision, time-series forecasting, dynamic graphs, and real-world risk applications, LeTE consistently outperforms hand-crafted encodings and traditional functional encodings, often achieving state-of-the-art results at lower embedding dimensions. The approach also offers interpretability by enabling reconstruction of the learned nonlinear mappings. Overall, LeTE provides a versatile, scalable, and interpretable time-encoding solution applicable to a wide range of temporal models and tasks.

Abstract

Effectively modeling time information and incorporating it into applications or models involving chronologically occurring events is crucial. Real-world scenarios often involve diverse and complex time patterns, which pose significant challenges for time encoding methods. While previous methods focus on capturing time patterns, many rely on specific inductive biases, such as using trigonometric functions to model periodicity. This narrow focus on single-pattern modeling makes them less effective in handling the diversity and complexities of real-world time patterns. In this paper, we investigate to improve the existing commonly used time encoding methods and introduce Learnable Transformation-based Generalized Time Encoding (LeTE). We propose using deep function learning techniques to parameterize non-linear transformations in time encoding, making them learnable and capable of modeling generalized time patterns, including diverse and complex temporal dynamics. By enabling learnable transformations, LeTE encompasses previous methods as specific cases and allows seamless integration into a wide range of tasks. Through extensive experiments across diverse domains, we demonstrate the versatility and effectiveness of LeTE.

Rethinking Time Encoding via Learnable Transformation Functions

TL;DR

This work addresses the limitation of fixed-pattern time encodings in real-world temporal data by introducing Learnable Transformation-based Generalized Time Encoding (LeTE). LeTE parameterizes nonlinear time transformations with learnable Fourier series or B-spline components, and combines them into a flexible, dimension-efficient embedding that generalizes prior encodings while remaining invariant to time rescaling. Across event-based vision, time-series forecasting, dynamic graphs, and real-world risk applications, LeTE consistently outperforms hand-crafted encodings and traditional functional encodings, often achieving state-of-the-art results at lower embedding dimensions. The approach also offers interpretability by enabling reconstruction of the learned nonlinear mappings. Overall, LeTE provides a versatile, scalable, and interpretable time-encoding solution applicable to a wide range of temporal models and tasks.

Abstract

Effectively modeling time information and incorporating it into applications or models involving chronologically occurring events is crucial. Real-world scenarios often involve diverse and complex time patterns, which pose significant challenges for time encoding methods. While previous methods focus on capturing time patterns, many rely on specific inductive biases, such as using trigonometric functions to model periodicity. This narrow focus on single-pattern modeling makes them less effective in handling the diversity and complexities of real-world time patterns. In this paper, we investigate to improve the existing commonly used time encoding methods and introduce Learnable Transformation-based Generalized Time Encoding (LeTE). We propose using deep function learning techniques to parameterize non-linear transformations in time encoding, making them learnable and capable of modeling generalized time patterns, including diverse and complex temporal dynamics. By enabling learnable transformations, LeTE encompasses previous methods as specific cases and allows seamless integration into a wide range of tasks. Through extensive experiments across diverse domains, we demonstrate the versatility and effectiveness of LeTE.
Paper Structure (59 sections, 3 theorems, 38 equations, 19 figures, 10 tables)

This paper contains 59 sections, 3 theorems, 38 equations, 19 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Functional Time Encodings
  4. Deep Function Learning
  5. Methods
  6. LeTE
  7. Properties of LeTE
  8. Use of Time Encoding
  9. Experiments
  10. Time as the Only Input
  11. Experiments on Time Series Tasks
  12. Experiments on Dynamic Graph Tasks
  13. Experiments on Real-World Application
  14. Dimensions of Time Encoding
  15. Additional Experiments
  16. ...and 44 more sections

Key Result

Proposition 2.1

Mathematically, with selected values for $\omega_i$ and $\varphi_i$, the aforementioned FTR and T2V can be unified into the following forms: Including the first dimension: Or excluding the first dimension: or

Figures (19)

  • Figure 1: A comparison of previous time encoding methods and proposed LeTE.
  • Figure 2: An illustration of Combined LeTE: the first dimension is parameterized by Fourier series expansion and the last dimension is parameterized by B-Splines.
  • Figure 3: Time as the only input: Comparison of time encodings on sequential MNIST.
  • Figure 4: Results evaluated by AUC-ROC, TP10 and Recall@10 on real business datasets.
  • Figure 5: Average Precision results comparing different dimensions of the FTE and Spline-based LeTE on Wikipedia/TGN and MOOC/TGN.
  • ...and 14 more figures

Theorems & Definitions (3)

  • Proposition 2.1
  • Proposition 3.1
  • Proposition 3.2