Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction

Kefan Wang, Hao Wang, Wei Guo, Yong Liu, Jianghao Lin, Defu Lian, Enhong Chen

TL;DR

This work tackles CTR prediction by addressing the limitations of purely explicit or implicit interaction modeling and the fusion gaps in two-stream architectures. It introduces Dynamic Low-Order-Aware Fusion (DLF), featuring a Residual-Aware Low-Order Interaction Network (RLI) to preserve low-order signals and a Network-Aware Attention Fusion Module (NAF) for layer-wise, dynamic integration of explicit and implicit representations. Through extensive experiments on multiple public datasets, DLF achieves state-of-the-art performance, especially on sparse data, and ablations confirm the complementary importance of RLI and NAF. The approach offers a flexible, extensible framework for sophisticated feature interaction modeling in CTR tasks, with code available publicly.

Abstract

Click-through rate (CTR) prediction is a critical task in online advertising and recommender systems, relying on effective modeling of feature interactions. Explicit interactions capture predefined relationships, such as inner products, but often suffer from data sparsity, while implicit interactions excel at learning complex patterns through non-linear transformations but lack inductive biases for efficient low-order modeling. Existing two-stream architectures integrate these paradigms but face challenges such as limited information sharing, gradient imbalance, and difficulty preserving low-order signals in sparse CTR data. We propose a novel framework, Dynamic Low-Order-Aware Fusion (DLF), which addresses these limitations through two key components: a Residual-Aware Low-Order Interaction Network (RLI) and a Network-Aware Attention Fusion Module (NAF). RLI explicitly preserves low-order signals while mitigating redundancy from residual connections, and NAF dynamically integrates explicit and implicit representations at each layer, enhancing information sharing and alleviating gradient imbalance. Together, these innovations balance low-order and high-order interactions, improving model expressiveness. Extensive experiments on public datasets demonstrate that DLF achieves state-of-the-art performance in CTR prediction, addressing key limitations of existing models. The implementation is publicly available at https://github.com/USTC-StarTeam/DLF.

DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction

TL;DR

This work tackles CTR prediction by addressing the limitations of purely explicit or implicit interaction modeling and the fusion gaps in two-stream architectures. It introduces Dynamic Low-Order-Aware Fusion (DLF), featuring a Residual-Aware Low-Order Interaction Network (RLI) to preserve low-order signals and a Network-Aware Attention Fusion Module (NAF) for layer-wise, dynamic integration of explicit and implicit representations. Through extensive experiments on multiple public datasets, DLF achieves state-of-the-art performance, especially on sparse data, and ablations confirm the complementary importance of RLI and NAF. The approach offers a flexible, extensible framework for sophisticated feature interaction modeling in CTR tasks, with code available publicly.

Abstract

Click-through rate (CTR) prediction is a critical task in online advertising and recommender systems, relying on effective modeling of feature interactions. Explicit interactions capture predefined relationships, such as inner products, but often suffer from data sparsity, while implicit interactions excel at learning complex patterns through non-linear transformations but lack inductive biases for efficient low-order modeling. Existing two-stream architectures integrate these paradigms but face challenges such as limited information sharing, gradient imbalance, and difficulty preserving low-order signals in sparse CTR data. We propose a novel framework, Dynamic Low-Order-Aware Fusion (DLF), which addresses these limitations through two key components: a Residual-Aware Low-Order Interaction Network (RLI) and a Network-Aware Attention Fusion Module (NAF). RLI explicitly preserves low-order signals while mitigating redundancy from residual connections, and NAF dynamically integrates explicit and implicit representations at each layer, enhancing information sharing and alleviating gradient imbalance. Together, these innovations balance low-order and high-order interactions, improving model expressiveness. Extensive experiments on public datasets demonstrate that DLF achieves state-of-the-art performance in CTR prediction, addressing key limitations of existing models. The implementation is publicly available at https://github.com/USTC-StarTeam/DLF.

Paper Structure

This paper contains 28 sections, 17 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Feature Interaction Modeling with Stacked Architectures
  4. Feature Interaction Modeling with Two-Stream Parallel Architectures
  5. Preliminary
  6. Problem Definition
  7. Feature Interaction in CTR Models
  8. Low-order interactions with the embedding layer.
  9. High-order interactions within the current layer.
  10. The gap in interaction order.
  11. Methodology
  12. Overview of the Proposed Method
  13. Residual-Aware Low-Order Interaction Network
  14. Network-Aware Attention Fusion Module
  15. Block-Wise Attention Mechanism
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Drawbacks of deep explicit interaction and deep implicit interaction
  • Figure 2: The overall framework of the proposed DLF model. The left part shows the complete workflow of DLF, with the core modules being the Residual-Aware Low-Order Interaction Network (RLI) and the Network-Aware Attention Fusion Module (NAF). The right part provides detailed visualizations of the three interaction blocks in RLI: the low-rank interaction block, the high-rank interaction block, and the implicit interaction block, highlighting their distinct computational mechanisms.
  • Figure 3: Interaction fusion methods analysis
  • Figure 4: Hyper-Parameter Analysis of DLF