Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting Customer Satisfaction by Replicating the Survey Response Distribution

Etienne Manderscheid, Matthias Lee

TL;DR

A method is introduced such that predicted CSAT (pCSAT) scores accurately replicate the distribution of survey CSAT responses for every call center with sufficient data in a live production environment and can be applied to many multiclass classification problems to improve the class balance.

Abstract

For many call centers, customer satisfaction (CSAT) is a key performance indicator (KPI). However, only a fraction of customers take the CSAT survey after the call, leading to a biased and inaccurate average CSAT value, and missed opportunities for coaching, follow-up, and rectification. Therefore, call centers can benefit from a model predicting customer satisfaction on calls where the customer did not complete the survey. Given that CSAT is a closely monitored KPI, it is critical to minimize any bias in the average predicted CSAT (pCSAT). In this paper, we introduce a method such that predicted CSAT (pCSAT) scores accurately replicate the distribution of survey CSAT responses for every call center with sufficient data in a live production environment. The method can be applied to many multiclass classification problems to improve the class balance and minimize its changes upon model updates.

Predicting Customer Satisfaction by Replicating the Survey Response Distribution

TL;DR

A method is introduced such that predicted CSAT (pCSAT) scores accurately replicate the distribution of survey CSAT responses for every call center with sufficient data in a live production environment and can be applied to many multiclass classification problems to improve the class balance.

Abstract

For many call centers, customer satisfaction (CSAT) is a key performance indicator (KPI). However, only a fraction of customers take the CSAT survey after the call, leading to a biased and inaccurate average CSAT value, and missed opportunities for coaching, follow-up, and rectification. Therefore, call centers can benefit from a model predicting customer satisfaction on calls where the customer did not complete the survey. Given that CSAT is a closely monitored KPI, it is critical to minimize any bias in the average predicted CSAT (pCSAT). In this paper, we introduce a method such that predicted CSAT (pCSAT) scores accurately replicate the distribution of survey CSAT responses for every call center with sufficient data in a live production environment. The method can be applied to many multiclass classification problems to improve the class balance and minimize its changes upon model updates.

Paper Structure

This paper contains 25 sections, 4 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Predicting Customer Satisfaction
  4. Replicating Class Distribution
  5. Data & Methods
  6. Transcripts
  7. Calls
  8. Trials
  9. Call Centers
  10. Threshold Optimization Procedure
  11. Model and Mapping
  12. Product Requirements
  13. Parameter Estimation
  14. Jointly estimate the four thresholds:
  15. Optimization Steps:
  16. ...and 10 more sections

Figures (5)

  • Figure 1: The mapping function that takes low CSAT probability ("proba") as input and outputs 1-5 pCSAT. In this example, the "proba" is 0.93, which is larger than $t_{1,2}$ so the model emits a pCSAT of 1.
  • Figure 2: The average loss for each of the five experimental conditions, binned by the call center's CSAT survey responses.
  • Figure 3: The average difference in percentage of satisfied calls between pCSAT and CSAT, broken down by the call center’s count of survey responses.
  • Figure 4: The average absolute difference between mean Predicted CSAT and survey CSAT for each of the five experimental conditions, binned by the call center's CSAT survey responses.
  • Figure 5: The Mean Squared Error measures the vector alignment between the normalized Predicted CSAT and survey CSAT distributions. Shown for each of the five experimental conditions, binned by the call center's CSAT survey responses.