eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data

TL;DR

eCeLLM tackles generalization gaps in e-commerce by instruction-tuning general-purpose LLMs on ECInstruct, a large-scale, real-world benchmark. The approach yields a family of eCeLLM models (L, M, S) that outperform GPT-4 Turbo, EcomGPT, and SoTA task-specific models on in-domain tasks and demonstrate strong out-of-domain and unseen-instruction generalization. The ECInstruct dataset provides broad coverage across 10 real-world e-commerce tasks with diverse, high-quality instructions and rigorous data splitting to evaluate IND and OOD performance. This work shows that open, large-scale instruction data can empower versatile, robust e-commerce foundation models with practical impact for cold-start scenarios and cross-task knowledge transfer.

Abstract

With tremendous efforts on developing effective e-commerce models, conventional e-commerce models show limited success in generalist e-commerce modeling, and suffer from unsatisfactory performance on new users and new products - a typical out-of-domain generalization challenge. Meanwhile, large language models (LLMs) demonstrate outstanding performance in generalist modeling and out-of-domain generalizability in many fields. Toward fully unleashing their power for e-commerce, in this paper, we construct ECInstruct, the first open-sourced, large-scale, and high-quality benchmark instruction dataset for e-commerce. Leveraging ECInstruct, we develop eCeLLM, a series of e-commerce LLMs, by instruction-tuning general-purpose LLMs. Our comprehensive experiments and evaluation demonstrate that eCeLLM models substantially outperform baseline models, including the most advanced GPT-4, and the state-of-the-art task-specific models in in-domain evaluation. Moreover, eCeLLM exhibits excellent generalizability to out-of-domain settings, including unseen products and unseen instructions, highlighting its superiority as a generalist e-commerce model. Both the ECInstruct dataset and the eCeLLM models show great potential in empowering versatile and effective LLMs for e-commerce. ECInstruct and eCeLLM models are publicly accessible through https://ninglab.github.io/eCeLLM.

Figures (5)

• Figure 1: Overall scheme of $\mathop{\mathtt{eCeLLM}}\limits$ instruction-tuned with $\mathop{\mathtt{ECInstruct}}\limits$
• Figure 2: IND Evaluation
• Figure 3: OOD Evaluation
• Figure A1: Distribution of Product Category
• Figure A2: Distribution of Input Length