Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Click-Through Rate Prediction Method Based on Cross-Importance of Multi-Order Features

Hao Wang, Nao Li

TL;DR

This work tackles the challenge of predicting click-through rate (CTR) in high-dimensional, sparse settings by learning the relative importance of multi-order feature crosses. It introduces FiiNet, which integrates Selective Kernel Networks (SKNet) to explicitly construct and weight cross-order interactions, followed by a deep neural network and a sigmoid output for CTR. Through extensive experiments on two real-world datasets, FiiNet achieves higher AUC and lower Logloss than a broad set of baselines, with ablation studies confirming the value of SKNet components and multi-order crosses. The approach also enhances interpretability by revealing which feature crosses carry the most predictive weight, offering practical benefits for recommender systems. The method is implemented in PyTorch for reproducibility and deployment.

Abstract

Most current click-through rate prediction(CTR)models create explicit or implicit high-order feature crosses through Hadamard product or inner product, with little attention to the importance of feature crossing; only few models are either limited to the second-order explicit feature crossing, implicitly to high-order feature crossing, or can learn the importance of high-order explicit feature crossing but fail to provide good interpretability for the model. This paper proposes a new model, FiiNet (Multiple Order Feature Interaction Importance Neural Networks). The model first uses the selective kernel network (SKNet) to explicitly construct multi-order feature crosses. It dynamically learns the importance of feature interaction combinations in a fine grained manner, increasing the attention weight of important feature cross combinations and reducing the weight of featureless crosses. To verify that the FiiNet model can dynamically learn the importance of feature interaction combinations in a fine-grained manner and improve the model's recommendation performance and interpretability, this paper compares it with many click-through rate prediction models on two real datasets, proving that the FiiNet model incorporating the selective kernel network can effectively improve the recommendation effect and provide better interpretability. FiiNet model implementations are available in PyTorch.

A Click-Through Rate Prediction Method Based on Cross-Importance of Multi-Order Features

TL;DR

This work tackles the challenge of predicting click-through rate (CTR) in high-dimensional, sparse settings by learning the relative importance of multi-order feature crosses. It introduces FiiNet, which integrates Selective Kernel Networks (SKNet) to explicitly construct and weight cross-order interactions, followed by a deep neural network and a sigmoid output for CTR. Through extensive experiments on two real-world datasets, FiiNet achieves higher AUC and lower Logloss than a broad set of baselines, with ablation studies confirming the value of SKNet components and multi-order crosses. The approach also enhances interpretability by revealing which feature crosses carry the most predictive weight, offering practical benefits for recommender systems. The method is implemented in PyTorch for reproducibility and deployment.

Abstract

Most current click-through rate prediction(CTR)models create explicit or implicit high-order feature crosses through Hadamard product or inner product, with little attention to the importance of feature crossing; only few models are either limited to the second-order explicit feature crossing, implicitly to high-order feature crossing, or can learn the importance of high-order explicit feature crossing but fail to provide good interpretability for the model. This paper proposes a new model, FiiNet (Multiple Order Feature Interaction Importance Neural Networks). The model first uses the selective kernel network (SKNet) to explicitly construct multi-order feature crosses. It dynamically learns the importance of feature interaction combinations in a fine grained manner, increasing the attention weight of important feature cross combinations and reducing the weight of featureless crosses. To verify that the FiiNet model can dynamically learn the importance of feature interaction combinations in a fine-grained manner and improve the model's recommendation performance and interpretability, this paper compares it with many click-through rate prediction models on two real datasets, proving that the FiiNet model incorporating the selective kernel network can effectively improve the recommendation effect and provide better interpretability. FiiNet model implementations are available in PyTorch.
Paper Structure (16 sections, 9 equations, 4 figures, 4 tables)

This paper contains 16 sections, 9 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Feature Crosses Model Related Work
  4. Selective kernel network method
  5. our proposed model-FiiNet
  6. Sparse Input Layer and Embedding Layer
  7. Selective kernel network Layer
  8. Deep Neural Network Layer
  9. Output Layer
  10. Experiment and result analysis
  11. Data sets and experiment setup
  12. Model performance analysis
  13. Embedding dimension analysis
  14. Ablation experiments
  15. Analysis of Feature Cross Importance
  16. ...and 1 more sections

Figures (4)

  • Figure 1: SKNet mechanism layer.
  • Figure 2: The index on the verification set varies with the embedding dimension.
  • Figure 3: Multi-index comparison results of four experimental models on the datasets.
  • Figure 4: Comparison of different weights of cross-combinations of multi-order features before and after training on the Book-Crossing dataset.