Making Large Language Models Better Knowledge Miners for Online Marketing with Progressive Prompting Augmentation

TL;DR

The paper addresses the challenge of mining a marketing-oriented knowledge graph (MoKG) using large language models, where pure LLM prompting often yields uncontrollable relations and high costs. It introduces PAIR, a framework that leverages prior knowledge, adaptive relation filtering, and progressive prompting augmentation for robust entity expansion, culminating in a reliable aggregation that accounts for self-consistency and semantic relatedness. A lightweight variant, LightPAIR, uses a strong teacher-LLM to generate a high-quality corpus for fine-tuning smaller models, enabling offline deployment suitable for large-scale marketing applications and audience targeting. Experiments show that PAIR outperforms baselines on accuracy, novelty, and diversity, and LightPAIR achieves comparable performance with significantly lower resource requirements, validating its practical applicability in real-world online marketing scenarios.

Abstract

Nowadays, the rapid development of mobile economy has promoted the flourishing of online marketing campaigns, whose success greatly hinges on the efficient matching between user preferences and desired marketing campaigns where a well-established Marketing-oriented Knowledge Graph (dubbed as MoKG) could serve as the critical "bridge" for preference propagation. In this paper, we seek to carefully prompt a Large Language Model (LLM) with domain-level knowledge as a better marketing-oriented knowledge miner for marketing-oriented knowledge graph construction, which is however non-trivial, suffering from several inevitable issues in real-world marketing scenarios, i.e., uncontrollable relation generation of LLMs,insufficient prompting ability of a single prompt, the unaffordable deployment cost of LLMs. To this end, we propose PAIR, a novel Progressive prompting Augmented mIning fRamework for harvesting marketing-oriented knowledge graph with LLMs. In particular, we reduce the pure relation generation to an LLM based adaptive relation filtering process through the knowledge-empowered prompting technique. Next, we steer LLMs for entity expansion with progressive prompting augmentation,followed by a reliable aggregation with comprehensive consideration of both self-consistency and semantic relatedness. In terms of online serving, we specialize in a small and white-box PAIR (i.e.,LightPAIR),which is fine-tuned with a high-quality corpus provided by a strong teacher-LLM. Extensive experiments and practical applications in audience targeting verify the effectiveness of the proposed (Light)PAIR.

Figures (8)

• Figure 1: An example of the marketing-oriented knowledge graph.
• Figure 2: Overall architecture of PAIR.
• Figure 3: The finetuning and inference pipeline of LightPAIR.
• Figure 4: Illustration of the knowledge graph enriched by the PAIR.
• Figure 5: Deployment of LightPAIR (i.e., "Offline A") for audience targeting and its comparison to the original EGL system with TRMP framework yang2023interested(i.e., "Offline B").