Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Augmented Mixture-of-Experts for QoS Prediction

Kecheng Cai, Chao Peng, Chenyang Xu, Xia Chen

TL;DR

QoS prediction is challenged by sparse user–service interactions. The paper introduces Self-Augmented Mixture-of-Experts (SA-MoE), which alternates between representation learning via matrix factorization and a Mixture-of-Experts predictor, using a refill mechanism and pseudo-labels to iteratively complete the interaction matrix and refine predictions. The main contributions are a refill-based self-augmentation that improves latent representations, a MoE architecture with expert specialization and emergent diversity, and a pseudo-label training regime that enhances generalization under extreme sparsity. Experiments on the WS-DREAM dataset demonstrate consistent gains over baselines for both response time and throughput metrics, validating robustness and practical impact in sparse QoS settings. The framework offers a general approach to robust sparse prediction in service-oriented contexts, enabling more reliable QoS forecasting for resource allocation and user experience optimization.

Abstract

Quality of Service (QoS) prediction is one of the most fundamental problems in service computing and personalized recommendation. In the problem, there is a set of users and services, each associated with a set of descriptive features. Interactions between users and services produce feedback values, typically represented as numerical QoS metrics such as response time or availability. Given the observed feedback for a subset of user-service pairs, the goal is to predict the QoS values for the remaining pairs. A key challenge in QoS prediction is the inherent sparsity of user-service interactions, as only a small subset of feedback values is typically observed. To address this, we propose a self-augmented strategy that leverages a model's own predictions for iterative refinement. In particular, we partially mask the predicted values and feed them back into the model to predict again. Building on this idea, we design a self-augmented mixture-of-experts model, where multiple expert networks iteratively and collaboratively estimate QoS values. We find that the iterative augmentation process naturally aligns with the MoE architecture by enabling inter-expert communication: in the second round, each expert receives the first-round predictions and refines its output accordingly. Experiments on benchmark datasets show that our method outperforms existing baselines and achieves competitive results.

Self-Augmented Mixture-of-Experts for QoS Prediction

TL;DR

QoS prediction is challenged by sparse user–service interactions. The paper introduces Self-Augmented Mixture-of-Experts (SA-MoE), which alternates between representation learning via matrix factorization and a Mixture-of-Experts predictor, using a refill mechanism and pseudo-labels to iteratively complete the interaction matrix and refine predictions. The main contributions are a refill-based self-augmentation that improves latent representations, a MoE architecture with expert specialization and emergent diversity, and a pseudo-label training regime that enhances generalization under extreme sparsity. Experiments on the WS-DREAM dataset demonstrate consistent gains over baselines for both response time and throughput metrics, validating robustness and practical impact in sparse QoS settings. The framework offers a general approach to robust sparse prediction in service-oriented contexts, enabling more reliable QoS forecasting for resource allocation and user experience optimization.

Abstract

Quality of Service (QoS) prediction is one of the most fundamental problems in service computing and personalized recommendation. In the problem, there is a set of users and services, each associated with a set of descriptive features. Interactions between users and services produce feedback values, typically represented as numerical QoS metrics such as response time or availability. Given the observed feedback for a subset of user-service pairs, the goal is to predict the QoS values for the remaining pairs. A key challenge in QoS prediction is the inherent sparsity of user-service interactions, as only a small subset of feedback values is typically observed. To address this, we propose a self-augmented strategy that leverages a model's own predictions for iterative refinement. In particular, we partially mask the predicted values and feed them back into the model to predict again. Building on this idea, we design a self-augmented mixture-of-experts model, where multiple expert networks iteratively and collaboratively estimate QoS values. We find that the iterative augmentation process naturally aligns with the MoE architecture by enabling inter-expert communication: in the second round, each expert receives the first-round predictions and refines its output accordingly. Experiments on benchmark datasets show that our method outperforms existing baselines and achieves competitive results.
Paper Structure (40 sections, 17 equations, 4 figures, 4 tables)

This paper contains 40 sections, 17 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Our main contributions are summarized as follows:
  3. Preliminary
  4. Problem Formulation
  5. Neural Network-Based Modeling
  6. Methodology
  7. Self-Augmented Representation Learning
  8. Collaborative Representation via MF.
  9. Prediction and Refill.
  10. Self-Augmentation Semantics.
  11. Expert Collaboration via MoE Architecture
  12. Expert Specialization and Iterative Communication.
  13. Emergent Expert Diversity.
  14. Self-Augmented Training with Pseudo Labels
  15. Implementation Details and Feature Fusion Strategy
  16. ...and 25 more sections

Figures (4)

  • Figure 1: The architecture of our approach, illustrating the matrix factorization module (left), the two-tower feature fusion module (middle) and the MoE module (right). In the MoE module, $\otimes$ denotes element-wise multiplication and $\oplus$ denotes element-wise addition.
  • Figure 2: Expert Specialization and MoE Aggregation in QoS Prediction
  • Figure 3: Impact of Matrix Refill Ratio ($r_m$) on MSE and RMSE for Different Densities
  • Figure 4: Impact of Matrix Refill Ratio ($r_m$) on MSE and RMSE for Different Densities