FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applications

TL;DR

FinTRec tackles unified sequential personalization and ads targeting in Financial Services by addressing long-range, multi-channel user signals and cross-product coordination under regulatory constraints. It introduces a transformer-based framework with a decoder-only pCTR and an encoder-only pCVR, fused with dynamic context $F_d$, static context $F_s$, and foundation-model embeddings $F_{fm}$, including product-adaptation fine-tuning via LoRA and new product embeddings. Off-line results show substantial gains in CVR likelihood and ranking quality, with ablations highlighting the importance of temporal encodings and FM embeddings, and product adaptation yielding significant recall improvements across FS products. Online A/B tests and offline simulations indicate strong production readiness, including latency targets of 120 ms at the 99th percentile and promising PV lifts, while also outlining limitations and future work toward unified architectures and same-day embeddings.

Abstract

Transformer-based architectures are widely adopted in sequential recommendation systems, yet their application in Financial Services (FS) presents distinct practical and modeling challenges for real-time recommendation. These include:a) long-range user interactions (implicit and explicit) spanning both digital and physical channels generating temporally heterogeneous context, b) the presence of multiple interrelated products require coordinated models to support varied ad placements and personalized feeds, while balancing competing business goals. We propose FinTRec, a transformer-based framework that addresses these challenges and its operational objectives in FS. While tree-based models have traditionally been preferred in FS due to their explainability and alignment with regulatory requirements, our study demonstrate that FinTRec offers a viable and effective shift toward transformer-based architectures. Through historic simulation and live A/B test correlations, we show FinTRec consistently outperforms the production-grade tree-based baseline. The unified architecture, when fine-tuned for product adaptation, enables cross-product signal sharing, reduces training cost and technical debt, while improving offline performance across all products. To our knowledge, this is the first comprehensive study of unified sequential recommendation modeling in FS that addresses both technical and business considerations.

Figures (6)

• Figure 1: Illustrative example of user's journey across channels, products and pages
• Figure 2: Yellow circle represent non$-$conversion events including impressions. Red circles represent conversions. For each conversion, item $\#1$ queries data back to $60$ days, while item $\#2$ and item $\#3$ queries data back to $30$ and $15$ days respectively. Then the data is temporally split for training/validation/test
• Figure 3: Illustrative example of intra-page tokenization. Tokens are grouped and arranged by their timestamps across pages
• Figure 4: Dynamic context ($F_d$) and Clickstream sequence ($S_u$) are processed to form sequential user representations ($\hat{S_u}$). Static ($F_s$) and FM embeddings ($F_{fm}$) are time-aligned with $\hat{S_u}$
• Figure 5: The pCVR model employs an encoder-only architecture (Figure \ref{['fig:encoder']}), while the pCTR model utilizes a causal decoder-only architecture (Figure \ref{['fig:decoder']}). Only positive samples are used for training pCTR, hence (t-2) is skipped from final loss calculation