Feature Fusion Revisited: Multimodal CTR Prediction for MMCTR Challenge

TL;DR

The paper tackles efficient multimodal CTR prediction by extending a DIN-based framework to fuse textual and visual signals with attention and feature-interaction modules, addressing the latency of large multimodal models. It presents a modular architecture with unified embeddings, multi-head target attention, SENet recalibration, cross-term interactions, and FiBiNET-inspired bilinear terms, evaluated across multiple multimodal item-embedding strategies. Among these, separately PCA-reduced BERT and CLIP embeddings concatenated (V4) delivered the best AUC (0.9306), highlighting the value of preserving modality-specific structure before fusion. The work demonstrates competitive performance in the EReL workshop (Task 2) and provides open-source code and pretrained weights, while outlining future directions in contrastive learning aligned with user-perceived similarity, advanced quantization, and data quality filtering to further boost robustness and efficiency.

Abstract

With the rapid advancement of Multimodal Large Language Models (MLLMs), an increasing number of researchers are exploring their application in recommendation systems. However, the high latency associated with large models presents a significant challenge for such use cases. The EReL@MIR workshop provided a valuable opportunity to experiment with various approaches aimed at improving the efficiency of multimodal representation learning for information retrieval tasks. As part of the competition's requirements, participants were mandated to submit a technical report detailing their methodologies and findings. Our team was honored to receive the award for Task 2 - Winner (Multimodal CTR Prediction). In this technical report, we present our methods and key findings. Additionally, we propose several directions for future work, particularly focusing on how to effectively integrate recommendation signals into multimodal representations. The codebase for our implementation is publicly available at: https://github.com/Lattice-zjj/MMCTR_Code, and the trained model weights can be accessed at: https://huggingface.co/FireFlyCourageous/MMCTR_DIN_MicroLens_1M_x1.