Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization

Zhaopeng Feng, Zijie Meng, Zuozhu Liu

TL;DR

This work tackles adapting large language models to e-commerce by building EC-Guide, a 74k-example instruction-tuning and quantization dataset across 24 subtasks and 5 task types. It combines an instruction-tuning pipeline with post-training quantization (GPTQ) and inference-time Chain-of-Thought to boost arithmetic tasks, yielding a model-agnostic solution that scales to larger systems. Evaluated on the Amazon KDD Cup'24 ShopBench tasks, the approach achieved 2nd place in Track 2 and 5th in Track 5, demonstrating practical effectiveness under constrained compute. The contribution lies in a comprehensive, domain-focused dataset and a scalable tuning-quantization-CoT workflow that can be applied to diverse e-commerce LLM deployments.

Abstract

Large language models (LLMs) have attracted considerable attention in various fields for their cost-effective solutions to diverse challenges, especially with advancements in instruction tuning and quantization. E-commerce, with its complex tasks and extensive product-user interactions, presents a promising application area for LLMs. However, the domain-specific concepts and knowledge inherent in e-commerce pose significant challenges for adapting general LLMs. To address this issue, we developed EC-Guide \href{https://github.com/fzp0424/EC-Guide-KDDUP-2024}, a comprehensive e-commerce guide for instruction tuning and quantization of LLMs. We also heuristically integrated Chain-of-Thought (CoT) during inference to enhance arithmetic performance. Our approach achieved the 2nd place in Track 2 and 5th place in Track 5 at the Amazon KDD Cup'24 \href{https://www.aicrowd.com/challenges/amazon-kdd-cup-2024-multi-task-online-shopping-challenge-for-llms}. Additionally, our solution is model-agnostic, enabling effective scalability across larger systems.

EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization

TL;DR

This work tackles adapting large language models to e-commerce by building EC-Guide, a 74k-example instruction-tuning and quantization dataset across 24 subtasks and 5 task types. It combines an instruction-tuning pipeline with post-training quantization (GPTQ) and inference-time Chain-of-Thought to boost arithmetic tasks, yielding a model-agnostic solution that scales to larger systems. Evaluated on the Amazon KDD Cup'24 ShopBench tasks, the approach achieved 2nd place in Track 2 and 5th in Track 5, demonstrating practical effectiveness under constrained compute. The contribution lies in a comprehensive, domain-focused dataset and a scalable tuning-quantization-CoT workflow that can be applied to diverse e-commerce LLM deployments.

Abstract

Large language models (LLMs) have attracted considerable attention in various fields for their cost-effective solutions to diverse challenges, especially with advancements in instruction tuning and quantization. E-commerce, with its complex tasks and extensive product-user interactions, presents a promising application area for LLMs. However, the domain-specific concepts and knowledge inherent in e-commerce pose significant challenges for adapting general LLMs. To address this issue, we developed EC-Guide \href{https://github.com/fzp0424/EC-Guide-KDDUP-2024}, a comprehensive e-commerce guide for instruction tuning and quantization of LLMs. We also heuristically integrated Chain-of-Thought (CoT) during inference to enhance arithmetic performance. Our approach achieved the 2nd place in Track 2 and 5th place in Track 5 at the Amazon KDD Cup'24 \href{https://www.aicrowd.com/challenges/amazon-kdd-cup-2024-multi-task-online-shopping-challenge-for-llms}. Additionally, our solution is model-agnostic, enabling effective scalability across larger systems.
Paper Structure (14 sections, 1 equation, 1 figure, 3 tables)

This paper contains 14 sections, 1 equation, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Dataset Construction
  5. Generation
  6. Multiple Choice Question
  7. Retrieval
  8. Ranking
  9. Named Entity Recognition
  10. Instruction Tuning
  11. Post Training Quantization
  12. Chain-of-Thought (CoT) Reasoning
  13. Experiments
  14. Conclusion

Figures (1)

  • Figure 1: Illustration of our solution. We first construct our EC-Guide (74k examples for 24 sub-tasks across 5 types) dataset from multiple sources ecinstructamazon-m2. Then we finetune Yi-1.5-34B young2024yi with QLoRA qlora and quantize it with GPTQ frantar2022gptq.