Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Chain-of-Thought Approach to Semantic Query Categorization in e-Commerce Taxonomies

Jetlir Duraj, Ishita Khan, Kilian Merkelbach, Mehran Elyasi

TL;DR

This work tackles the challenge of mapping user queries to semantically relevant leaf categories within large e-commerce taxonomies, addressing bias and sparsity in demand signals. It introduces a Chain-of-Thought Breadth-First-Search (CoT BFS) framework that integrates iterative LLM-based semantic scoring with a tree-search over taxonomy leaves, including context injection to reflect user intents and brand origins. The approach is evaluated against a k-NN baseline using human judgments, an AI pseudo-reference, and retrieval tests, demonstrating superior F1, precision, and recall, and revealing taxonomy gaps for targeted improvements. The methodology offers practical impact for improving search relevance and navigation at scale, while providing diagnostic tools to refine and expand taxonomies; scalability refinements (CoT-k-NN BFS and leaf-based k-NN + LLM) enable deployment across millions of queries.

Abstract

Search in e-Commerce is powered at the core by a structured representation of the inventory, often formulated as a category taxonomy. An important capability in e-Commerce with hierarchical taxonomies is to select a set of relevant leaf categories that are semantically aligned with a given user query. In this scope, we address a fundamental problem of search query categorization in real-world e-Commerce taxonomies. A correct categorization of a query not only provides a way to zoom into the correct inventory space, but opens the door to multiple intent understanding capabilities for a query. A practical and accurate solution to this problem has many applications in e-commerce, including constraining retrieved items and improving the relevance of the search results. For this task, we explore a novel Chain-of-Thought (CoT) paradigm that combines simple tree-search with LLM semantic scoring. Assessing its classification performance on human-judged query-category pairs, relevance tests, and LLM-based reference methods, we find that the CoT approach performs better than a benchmark that uses embedding-based query category predictions. We show how the CoT approach can detect problems within a hierarchical taxonomy. Finally, we also propose LLM-based approaches for query-categorization of the same spirit, but which scale better at the range of millions of queries.

A Chain-of-Thought Approach to Semantic Query Categorization in e-Commerce Taxonomies

TL;DR

This work tackles the challenge of mapping user queries to semantically relevant leaf categories within large e-commerce taxonomies, addressing bias and sparsity in demand signals. It introduces a Chain-of-Thought Breadth-First-Search (CoT BFS) framework that integrates iterative LLM-based semantic scoring with a tree-search over taxonomy leaves, including context injection to reflect user intents and brand origins. The approach is evaluated against a k-NN baseline using human judgments, an AI pseudo-reference, and retrieval tests, demonstrating superior F1, precision, and recall, and revealing taxonomy gaps for targeted improvements. The methodology offers practical impact for improving search relevance and navigation at scale, while providing diagnostic tools to refine and expand taxonomies; scalability refinements (CoT-k-NN BFS and leaf-based k-NN + LLM) enable deployment across millions of queries.

Abstract

Search in e-Commerce is powered at the core by a structured representation of the inventory, often formulated as a category taxonomy. An important capability in e-Commerce with hierarchical taxonomies is to select a set of relevant leaf categories that are semantically aligned with a given user query. In this scope, we address a fundamental problem of search query categorization in real-world e-Commerce taxonomies. A correct categorization of a query not only provides a way to zoom into the correct inventory space, but opens the door to multiple intent understanding capabilities for a query. A practical and accurate solution to this problem has many applications in e-commerce, including constraining retrieved items and improving the relevance of the search results. For this task, we explore a novel Chain-of-Thought (CoT) paradigm that combines simple tree-search with LLM semantic scoring. Assessing its classification performance on human-judged query-category pairs, relevance tests, and LLM-based reference methods, we find that the CoT approach performs better than a benchmark that uses embedding-based query category predictions. We show how the CoT approach can detect problems within a hierarchical taxonomy. Finally, we also propose LLM-based approaches for query-categorization of the same spirit, but which scale better at the range of millions of queries.
Paper Structure (22 sections, 3 figures, 5 tables)

This paper contains 22 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Demand-based approaches to category prediction
  4. Semantic-based methods and query understanding
  5. LLMs
  6. Taxonomy integration
  7. Methodology
  8. High level overview
  9. Breadth first search design (CoT BFS)
  10. Scalable approaches for CoT and LLM scoring
  11. CoT-k-NN hybrid BFS
  12. k-NN-search + LLM scoring on leaf categories
  13. Experimentation
  14. Baseline model: k-nearest neighbors categorization
  15. Evaluation against baseline
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Overview of our system architecture. First, we build our category tree and enrich it with category descriptions. Then, we search within the tree for the node that best fits the search query. Finally, diagnostics and refinements of our taxonomy can be extracted.
  • Figure 2: CoT BFS output for the query acoustic guitar. Mixtral-8x7B was used as the LLM. The prompt contains information about the path from the root of the tree to the current node, the immediate parent node, and the child node to be rated.
  • Figure 3: CoT BFS output for the query canon camera with accessory intent. The query intent being supplied as context guides the category search into categories that correspond to accessories for the entity in the query.