Hyena Operator for Fast Sequential Recommendation

Abstract

Sequential recommendation models, particularly those based on attention, achieve strong accuracy but incur quadratic complexity, making long user histories prohibitively expensive. Sub-quadratic operators such as Hyena provide efficient alternatives in language modeling, but their potential in recommendation remains underexplored. We argue that Hyena faces challenges in recommendation due to limited representation capacity on sparse, long user sequences. To address these challenges, we propose HyenaRec, a novel sequential recommender that integrates polynomial-based kernel parameterization with gated convolutions. Specifically, we design convolutional kernels using Legendre orthogonal polynomials, which provides a smooth and compact basis for modeling long-term temporal dependencies. A complementary gating mechanism captures fine-grained short-term behavioral bursts, yielding a hybrid architecture that balances global temporal evolution with localized user interests under sparse feedback. This construction enhances expressiveness while scaling linearly with sequence length. Extensive experiments on multiple real-world datasets demonstrate that HyenaRec consistently outperforms Attention-, Recurrent-, and other baselines in ranking accuracy. Moreover, it trains significantly faster (up to 6x speedup), with particularly pronounced advantages on long-sequence scenarios where efficiency is maintained without sacrificing accuracy. These results highlight polynomial-based kernel parameterization as a principled and scalable alternative to attention for sequential recommendation.

Figures (5)

• Figure 1: Overall Architecture of HyenaRec, depicting the input & embedding layer, Hyena-based sequential backbone (where each Hyena Block contains a Hyena Operator that utilizes Legendre Hyena Filters for long-range modeling), and the Top-K prediction layer for next-item recommendation.
• Figure 2: Visualization of the LegendreHyenaFilter. (a) Legendre polynomial basis functions $P_n(x)$($n = 0,1,2,3$) exhibit orthogonality and stable boundary behavior. (b) A coefficient vector$\mathbf{c}_d$ (for a single channel) linearly weights these bases to construct channel-specific convolutional kernels. (c) Comparison between kernels from LegendreHyenaFilter (smoother, more stable) and original Hyena, emphasizing suitability for sparse, variable-length recommendation sequences.
• Figure 3: Training time ratios (TTR) of baseline models relative to HyenaRec, which is normalized to 1×. Each bar shows the relative training cost of a model on a given dataset.
• Figure 4: Hyperparameter sensitivity of HyenaRec.
• Figure 5: Performance of HyenaRec under different Legendre filter orders $K$. Left: ML-1M dataset; Right: Video dataset. Both Recall@10 and NDCG@10 are shown.