Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction

TL;DR

This work addresses length-induced bias in CTR prediction by explicitly conditioning on behavior sequence length. The proposed Length-Adaptive Interest Network (LAIN) introduces three components—Spectral Length Encoder, Length-Conditioned Prompting, and Length-Modulated Attention—to adapt representations and attention mechanisms to sequence length, mitigating attention polarization and length-signal deficiency. Across three real-world datasets and multiple backbones, LAIN yields consistent improvements, notably boosting short-sequence accuracy (e.g., up to +1.08% AUC for 0–100 length) while preserving long-sequence performance, with gains such as up to +1.2% GAUC and −1.6% LogLoss. The framework is lightweight, plug-and-play, and broadly applicable to existing CTR architectures, offering a practical solution to length-induced bias in large-scale recommendation systems.

Abstract

User behavior sequences in modern recommendation systems exhibit significant length heterogeneity, ranging from sparse short-term interactions to rich long-term histories. While longer sequences provide more context, we observe that increasing the maximum input sequence length in existing CTR models paradoxically degrades performance for short-sequence users due to attention polarization and length imbalance in training data. To address this, we propose LAIN(Length-Adaptive Interest Network), a plug-and-play framework that explicitly incorporates sequence length as a conditioning signal to balance long- and short-sequence modeling. LAIN consists of three lightweight components: a Spectral Length Encoder that maps length into continuous representations, Length-Conditioned Prompting that injects global contextual cues into both long- and short-term behavior branches, and Length-Modulated Attention that adaptively adjusts attention sharpness based on sequence length. Extensive experiments on three real-world benchmarks across five strong CTR backbones show that LAIN consistently improves overall performance, achieving up to 1.15% AUC gain and 2.25% log loss reduction. Notably, our method significantly improves accuracy for short-sequence users without sacrificing longsequence effectiveness. Our work offers a general, efficient, and deployable solution to mitigate length-induced bias in sequential recommendation.

Figures (4)

• Figure 1: (a) The effect of varying the maximum input sequence length on average AUC across five CTR models on the EBNeRD-small dataset. Longer sequences improve performance for users with extensive behavior histories ($>200$), while consistently degrading it for users with fewer interactions ($<100$). (b) The user behavior length distribution reveals a skewed pattern: short-sequence users form the majority, yet long-sequence users dominate the training sample.
• Figure 2: Attention polarization trend in baseline CTR models. Gini coefficient increases dramatically with sequence length, demonstrating progressive attention concentration that particularly affects short-sequence users.
• Figure 3: Overview of the Length-Adaptive Interest Network (LAIN) for CTR prediction. LAIN conditions on sequence length via a Spectral Length Encoder (SLE), which generates a continuous embedding $\mathbf{h}_{\text{len}}$ from the raw length $L$. This embedding modulates two components: Length-Conditioned Prompting (LCP), which prepends length-aware prompt tokens to the behavior sequence, and Length-Modulated Attention (LMA), which adjusts attention via query/key conditioning and dynamic temperature scaling. The resulting sequence representation is fused with other features for final CTR prediction.
• Figure 4: Parameter sensitivity analysis for AUC performance. All hyperparameter configurations consistently outperform the baseline (red dashed line), demonstrating the robustness of our approach across different parameter settings.