Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval

Xiaojie Sun, Keping Bi, Jiafeng Guo, Sihui Yang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, Xueqi Cheng

TL;DR

This work tackles dense retrieval on structured data by leveraging multi-granularity aspect information. It introduces MUlti-granulaRity-aware Aspect Learning (MURAL), which inserts dedicated aspect embeddings before content and after CLS, learns aspect representations through MLM-guided and group-wise contrastive objectives, and fuses multiple aspect signals with a CLS-Gating mechanism. By exploring three grouping strategies and offering both shared and unshared value embeddings, MURAL achieves state-of-the-art results on MA-Amazon and Alipay, with significant gains over baselines and notable effectiveness even without explicit aspect annotations. The study demonstrates that combining coarse and fine-grained aspect values improves semantic relationships among values and enhances retrieval performance, providing practical insights for multi-aspect dense retrieval systems.

Abstract

Dense retrieval methods have been mostly focused on unstructured text and less attention has been drawn to structured data with various aspects, e.g., products with aspects such as category and brand. Recent work has proposed two approaches to incorporate the aspect information into item representations for effective retrieval by predicting the values associated with the item aspects. Despite their efficacy, they treat the values as isolated classes (e.g., "Smart Homes", "Home, Garden & Tools", and "Beauty & Health") and ignore their fine-grained semantic relation. Furthermore, they either enforce the learning of aspects into the CLS token, which could confuse it from its designated use for representing the entire content semantics, or learn extra aspect embeddings only with the value prediction objective, which could be insufficient especially when there are no annotated values for an item aspect. Aware of these limitations, we propose a MUlti-granulaRity-aware Aspect Learning model (MURAL) for multi-aspect dense retrieval. It leverages aspect information across various granularities to capture both coarse and fine-grained semantic relations between values. Moreover, MURAL incorporates separate aspect embeddings as input to transformer encoders so that the masked language model objective can assist implicit aspect learning even without aspect-value annotations. Extensive experiments on two real-world datasets of products and mini-programs show that MURAL outperforms state-of-the-art baselines significantly.

A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval

TL;DR

This work tackles dense retrieval on structured data by leveraging multi-granularity aspect information. It introduces MUlti-granulaRity-aware Aspect Learning (MURAL), which inserts dedicated aspect embeddings before content and after CLS, learns aspect representations through MLM-guided and group-wise contrastive objectives, and fuses multiple aspect signals with a CLS-Gating mechanism. By exploring three grouping strategies and offering both shared and unshared value embeddings, MURAL achieves state-of-the-art results on MA-Amazon and Alipay, with significant gains over baselines and notable effectiveness even without explicit aspect annotations. The study demonstrates that combining coarse and fine-grained aspect values improves semantic relationships among values and enhances retrieval performance, providing practical insights for multi-aspect dense retrieval systems.

Abstract

Dense retrieval methods have been mostly focused on unstructured text and less attention has been drawn to structured data with various aspects, e.g., products with aspects such as category and brand. Recent work has proposed two approaches to incorporate the aspect information into item representations for effective retrieval by predicting the values associated with the item aspects. Despite their efficacy, they treat the values as isolated classes (e.g., "Smart Homes", "Home, Garden & Tools", and "Beauty & Health") and ignore their fine-grained semantic relation. Furthermore, they either enforce the learning of aspects into the CLS token, which could confuse it from its designated use for representing the entire content semantics, or learn extra aspect embeddings only with the value prediction objective, which could be insufficient especially when there are no annotated values for an item aspect. Aware of these limitations, we propose a MUlti-granulaRity-aware Aspect Learning model (MURAL) for multi-aspect dense retrieval. It leverages aspect information across various granularities to capture both coarse and fine-grained semantic relations between values. Moreover, MURAL incorporates separate aspect embeddings as input to transformer encoders so that the masked language model objective can assist implicit aspect learning even without aspect-value annotations. Extensive experiments on two real-world datasets of products and mini-programs show that MURAL outperforms state-of-the-art baselines significantly.
Paper Structure (23 sections, 10 equations, 4 figures, 7 tables)

This paper contains 23 sections, 10 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Preliminaries
  4. METHODOLOGY
  5. Aspect Representations
  6. Aspect Learning
  7. Multi-Granularity-Aspect Grouping
  8. Aspect Embedding Fusion
  9. Training Objectives
  10. EXPERIMENTAL SETTINGS
  11. Datasets
  12. Baselines
  13. Implementation Details
  14. Pre-training
  15. Fine-tuning
  16. ...and 8 more sections

Figures (4)

  • Figure 1: An example of a query and its candidate items.
  • Figure 2: SOTA multi-aspect dense retrieval models.
  • Figure 3: Our MURAL with single-objective-based grouping in a simplistic scenario of two aspects and two granularities.
  • Figure 4: The t-SNE plot of item representations for MADRAL and MURAL on MA-Amazon.