GTS-LUM: Reshaping User Behavior Modeling with LLMs in Telecommunications Industry
Liu Shi, Tianwu Zhou, Wei Xu, Li Liu, Zhexin Cui, Shaoyi Liang, Haoxing Niu, Yichong Tian, Jianwei Guo
TL;DR
GTS-LUM introduces a multi-modal encoder-adapter-LLM decoder framework tailored for telecommunications, addressing long-term and periodic user behavior with diverse time granularities. Key innovations include a telecom-specific timestamp processor, spectral clustering-based semantic IDs, multi-modal encoders for tables and graphs, and a Q-former that aligns semantic and business signals, plus a front-placed target-aware mechanism to blend history with the target. The model is trained in two phases, combining Q-former pretraining with sequence-text alignment tasks and end-to-end contrastive learning with a frozen LLM decoder. Experiments on industrial data show GTS-LUM surpasses LLM4Rec baselines, demonstrating strong practical impact for churn prediction, package recommendations, and marketing interventions in telecom contexts.
Abstract
As telecommunication service providers shifting their focus to analyzing user behavior for package design and marketing interventions, a critical challenge lies in developing a unified, end-to-end framework capable of modeling long-term and periodic user behavior sequences with diverse time granularities, multi-modal data inputs, and heterogeneous labels. This paper introduces GTS-LUM, a novel user behavior model that redefines modeling paradigms in telecommunication settings. GTS-LUM adopts a (multi-modal) encoder-adapter-LLM decoder architecture, enhanced with several telecom-specific innovations. Specifically, the model incorporates an advanced timestamp processing method to handle varying time granularities. It also supports multi-modal data inputs -- including structured tables and behavior co-occurrence graphs -- and aligns these with semantic information extracted by a tokenizer using a Q-former structure. Additionally, GTS-LUM integrates a front-placed target-aware mechanism to highlight historical behaviors most relevant to the target. Extensive experiments on industrial dataset validate the effectiveness of this end-to-end framework and also demonstrate that GTS-LUM outperforms LLM4Rec approaches which are popular in recommendation systems, offering an effective and generalizing solution for user behavior modeling in telecommunications.