SampleLLM: Optimizing Tabular Data Synthesis in Recommendations

TL;DR

SampleLLM addresses the challenge of generating high-quality synthetic tabular data for recommender systems by combining few-shot LLM generation with a distribution-alignment stage based on feature attribution. It introduces two stages: (i) instruction-refined, diverse-exemplar LLM data generation to approximate the target distribution, and (ii) an importance-sampling step using a semi-independent feature-interaction framework to align feature relationships and reduce distribution bias. The approach demonstrates improved augmentation utility and downstream performance across multiple datasets and even shows promise in online deployment, outperforming several state-of-the-art tabular synthesis methods. The work highlights the practical impact of integrating semantic understanding from LLMs with principled distribution alignment for data-scarce recommendation scenarios and beyond.

Abstract

Tabular data synthesis is crucial in machine learning, yet existing general methods-primarily based on statistical or deep learning models-are highly data-dependent and often fall short in recommender systems. This limitation arises from their difficulty in capturing complex distributions and understanding feature relationships from sparse and limited data, along with their inability to grasp semantic feature relations. Recently, Large Language Models (LLMs) have shown potential in generating synthetic data samples through few-shot learning and semantic understanding. However, they often suffer from inconsistent distribution and lack of diversity due to their inherent distribution disparity with the target dataset. To address these challenges and enhance tabular data synthesis for recommendation tasks, we propose a novel two-stage framework named SampleLLM to improve the quality of LLM-based tabular data synthesis for recommendations by ensuring better distribution alignment. In the first stage, SampleLLM employs LLMs with Chain-of-Thought prompts and diverse exemplars to generate data that closely aligns with the target dataset distribution, even when input samples are limited. The second stage uses an advanced feature attribution-based importance sampling method to refine feature relationships within the synthesized data, reducing any distribution biases introduced by the LLM. Experimental results on three recommendation datasets, two general datasets, and online deployment illustrate that SampleLLM significantly surpasses existing methods for recommendation tasks and holds promise for a broader range of tabular data scenarios.

Figures (7)

• Figure 1: Visualization of LLM-generated and original tabular samples on the HELOC dataset reveals that the synthetic data lacks distribution alignment and diversity, clustering around a few centers within the original data distribution.
• Figure 2: The overall structure of SampleLLM. (a) In the first stage, a manually designed instruction and $a$ samples extracted with clustering sampling are used as inputs to the LLM, which generates $b$ synthetic samples. This process is repeated $Q$ times. (b) In the second stage, a novel feature attribution-based importance sampling method is employed on the synthetic samples.
• Figure 3: A simplified instruction example.
• Figure 4: Ablation study of augmentation utility on Douban.
• Figure 5: Visualization analysis of the ablation study.