Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation

TL;DR

This paper addresses the gap between dense behavioral feedback and sparse explicit satisfaction signals in short-video recommendations by proposing EASQ, an end-to-end online framework that aligns rankings with true user satisfaction. The method builds a decoupled alignment pathway using a lightweight LoRA module and a multi-task MoE to separately model questionnaire supervision, while applying an online DPO objective to keep the main model aligned with sparse feedback in real time. Through extensive offline evaluation and large-scale online A/B testing, EASQ demonstrates consistent improvements in satisfaction-related metrics and engagement across scenarios, and has been successfully deployed in production with measurable business gains. The work provides a practical pathway for incorporating high-quality, sparse satisfaction signals into continual online learning systems without destabilizing the backbone, enabling more faithful optimization of user experience.

Abstract

Short-video recommender systems typically optimize ranking models using dense user behavioral signals, such as clicks and watch time. However, these signals are only indirect proxies of user satisfaction and often suffer from noise and bias. Recently, explicit satisfaction feedback collected through questionnaires has emerged as a high-quality direct alignment supervision, but is extremely sparse and easily overwhelmed by abundant behavioral data, making it difficult to incorporate into online recommendation models. To address these challenges, we propose a novel framework which is towards End-to-End Alignment of user Satisfaction via Questionaire, named EASQ, to enable real-time alignment of ranking models with true user satisfaction. Specifically, we first construct an independent parameter pathway for sparse questionnaire signals by combining a multi-task architecture and a lightweight LoRA module. The multi-task design separates sparse satisfaction supervision from dense behavioral signals, preventing the former from being overwhelmed. The LoRA module pre-inject these preferences in a parameter-isolated manner, ensuring stability in the backbone while optimizing user satisfaction. Furthermore, we employ a DPO-based optimization objective tailored for online learning, which aligns the main model outputs with sparse satisfaction signals in real time. This design enables end-to-end online learning, allowing the model to continuously adapt to new questionnaire feedback while maintaining the stability and effectiveness of the backbone. Extensive offline experiments and large-scale online A/B tests demonstrate that EASQ consistently improves user satisfaction metrics across multiple scenarios. EASQ has been successfully deployed in a production short-video recommendation system, delivering significant and stable business gains.

Figures (6)

• Figure 1: From Conventional Ranking Optimization to Online Learning with User Satisfaction. (a) Conventional online learning framework. (b) Proposed EASQ framework that aligns recommendations with sparse satisfaction feedback.
• Figure 2: Illustration of questionnaire triggering and implementation.
• Figure 3: Correlation Between Questionnaire Feedback and Posterior User Behavioral Signals.
• Figure 4: Top-K Softmax vs. ReLU Expert Routing.
• Figure 5: Performance for Ablation Study on Scenario#2.