Feature-aware Modulation for Learning from Temporal Tabular Data

TL;DR

The paper tackles temporal distribution shifts in tabular data by identifying evolving feature semantics as a key drift source. It introduces a feature-aware temporal modulation mechanism that conditions feature representations on temporal context via learnable transformations of distributional statistics, enabling semantic alignment over time. The approach balances generalization and adaptability, enabling both distributional and temporal extrapolation with low computational overhead. Empirical results on the TabReD benchmark show consistent improvements over static models and temporal-embedding baselines, with gains driven by modulation at multiple network layers and especially at the input level. This work offers a scalable, interpretable strategy for robust learning in non-stationary tabular environments.

Abstract

While tabular machine learning has achieved remarkable success, temporal distribution shifts pose significant challenges in real-world deployment, as the relationships between features and labels continuously evolve. Static models assume fixed mappings to ensure generalization, whereas adaptive models may overfit to transient patterns, creating a dilemma between robustness and adaptability. In this paper, we analyze key factors essential for constructing an effective dynamic mapping for temporal tabular data. We discover that evolving feature semantics-particularly objective and subjective meanings-introduce concept drift over time. Crucially, we identify that feature transformation strategies are able to mitigate discrepancies in feature representations across temporal stages. Motivated by these insights, we propose a feature-aware temporal modulation mechanism that conditions feature representations on temporal context, modulating statistical properties such as scale and skewness. By aligning feature semantics across time, our approach achieves a lightweight yet powerful adaptation, effectively balancing generalizability and adaptability. Benchmark evaluations validate the effectiveness of our method in handling temporal shifts in tabular data.

Figures (8)

• Figure 1: Left: Static models assume a time-invariant mapping $f$ for all temporal subset ${\mathcal{D}}_t$, while adaptive model $f_t$ dynamically adjust to each temporal stage. Right: Raw salary and location values (top) encode shifting subjective concepts like "high income" or "prime location," which vary across time. Our method aligns these semantics (bottom) by modulating feature distributions—using temporal statistics (e.g., mean, std, skewness)—to preserve concept consistency over time.
• Figure 2: Top: Empirical feature distributions over time, with colors ranging from dark (early periods) to bright (recent periods), exhibit clear non-stationarity in bias (left), scale (middle), and skewness (right). Bottom: Schematic illustration of learnable transformations applied to feature distributions: shifting the mean ($\beta$) aligns bias (left), adjusting standard deviation ($\gamma$) alters scale (middle), and modulating asymmetry ($\lambda$) reshapes skewness (right). These transformations enable semantic alignment across temporal stages, thereby strengthening both generalization and adaptability.
• Figure 3: Overview of our feature-aware temporal modulation framework. Temporal modulation can be applied on raw feature input, intermediate representation, and output logits. The modulator conditions temporal context $\psi(t)$ to predict parameter $\gamma, \beta, \lambda$ for modulation based on \ref{['eq:modulation']}.
• Figure 4: Pilot study on aligning feature semantics. The left plot illustrates the decision boundary learned by a static MLP, highlighting that such methods struggle to capture separability under temporal shifts. The top panel visualizes the evolving decision boundaries learned by an MLP with our temporal modulation, where modulation is applied once at the input layer. Each of the five subplots corresponds to a different temporal segment, revealing how the model adapts its decision boundary in response to temporal dynamics. The middle panel displays feature distributions after modulation, which are better aligned across time. This alignment enables the model to form a consistent decision boundary, as shown in the bottom panel. These results demonstrates that our lightweight modulation mechanism effectively aligns feature semantics, allowing the backbone network to operate within a unified conceptual space over time.
• Figure 5: Improvement in performance with temporal modulation. The bar chart compares the percentage improvement in performance across different models: MLP, MLP-PLR, and TabM. The left side shows results with temporal embeddings, while the right side demonstrates the improvement using our proposed temporal modulation method. Temporal modulation yields significant improvements, particularly for the MLP model, achieving a 2.09% increase, while other models exhibit more moderate gains.