Enhancing Spatio-temporal Quantile Forecasting with Curriculum Learning: Lessons Learned

TL;DR

This work presented an innovative paradigm that incorporates three separate forms of curriculum learning specifically targeting from spatial, temporal, and quantile perspectives specifically targeting from spatial, temporal, and quantile perspectives and demonstrated the effectiveness with extensive empirical evaluations.

Abstract

Training models on spatio-temporal (ST) data poses an open problem due to the complicated and diverse nature of the data itself, and it is challenging to ensure the model's performance directly trained on the original ST data. While limiting the variety of training data can make training easier, it can also lead to a lack of knowledge and information for the model, resulting in a decrease in performance. To address this challenge, we presented an innovative paradigm that incorporates three separate forms of curriculum learning specifically targeting from spatial, temporal, and quantile perspectives. Furthermore, our framework incorporates a stacking fusion module to combine diverse information from three types of curriculum learning, resulting in a strong and thorough learning process. We demonstrated the effectiveness of this framework with extensive empirical evaluations, highlighting its better performance in addressing complex ST challenges. We provided thorough ablation studies to investigate the effectiveness of our curriculum and to explain how it contributes to the improvement of learning efficiency on ST data.

Figures (7)

• Figure 1: Spatio-temporal problems often show a clear separation between easy and difficult instances, closely mirroring the curriculum learning approach that progresses from simple to complex tasks. This similarity suggests that curriculum learning's gradual difficulty increase is well-suited for addressing the varied complexities of spatio-temporal forecasting.
• Figure 2: The overview of our spatio-temporal-quantile curriculum learning framework. First, ST data is trained to get an initial model for initializing the curriculum learning. Subsequently, the three types of curriculum learning schedulers proposed in this work are applied. The knowledge from the three curriculum learning experts is then fused using a simple linear layer. The details of each type of curriculum learning are illustrated in the latter part of the figure.
• Figure 3: Case study on PEMS04.
• Figure 4: Performance comparison with different step sizes.
• Figure 5: Ablation experiments of Early Model. Point predictions are evaluated using RMSE and quantile predictions are gauged with the average of Q10, Q50, and Q90.