Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

e-Profits: A Business-Aligned Evaluation Metric for Profit-Sensitive Customer Churn Prediction

Awais Manzoor, M. Atif Qureshi, Etain Kidney, Luca Longo

TL;DR

This work addresses the mismatch between traditional churn-prediction metrics and financial impact by introducing e-Profits, a business-aligned evaluation that ties customer-level value, retention probabilities from Kaplan-Meier survival analysis, and intervention costs to per-customer profits. It enables model comparison without altering learning algorithms, and demonstrates through two telecom datasets that profit-driven ranking diverges from AUC/F1-based rankings, highlighting models that maximize ROI for both overall and high-value segments. The approach yields statistically significant profit gains and practical managerial insights, such as prioritizing models for budget-constrained campaigns and segment-targeting strategies. Overall, e-Profits bridges predictive modeling and profit-driven decision-making, offering a transparent, plug-in evaluation tool for churn management and beyond.

Abstract

Retention campaigns in customer relationship management often rely on churn prediction models evaluated using traditional metrics such as AUC and F1-score. However, these metrics fail to reflect financial outcomes and may mislead strategic decisions. We introduce e-Profits, a novel business-aligned evaluation metric that quantifies model performance based on customer lifetime value, retention probability, and intervention costs. Unlike existing profit-based metrics such as Expected Maximum Profit, which assume fixed population-level parameters, e-Profits uses Kaplan-Meier survival analysis to estimate tenure-conditioned (customer-level) one-period retention probabilities and supports granular, per-customer profit evaluation. We benchmark six classifiers across two telecom datasets (IBM Telco and Maven Telecom) and demonstrate that e-Profits reshapes model rankings compared to traditional metrics, revealing financial advantages in models previously overlooked by AUC or F1-score. The metric also enables segment-level insight into which models maximise return on investment for high-value customers. e-Profits provides a transparent, customer-level evaluation framework that bridges predictive modelling and profit-driven decision-making in operational churn management. All source code is available at: https://github.com/Awaismanzoor/eprofits.

e-Profits: A Business-Aligned Evaluation Metric for Profit-Sensitive Customer Churn Prediction

TL;DR

This work addresses the mismatch between traditional churn-prediction metrics and financial impact by introducing e-Profits, a business-aligned evaluation that ties customer-level value, retention probabilities from Kaplan-Meier survival analysis, and intervention costs to per-customer profits. It enables model comparison without altering learning algorithms, and demonstrates through two telecom datasets that profit-driven ranking diverges from AUC/F1-based rankings, highlighting models that maximize ROI for both overall and high-value segments. The approach yields statistically significant profit gains and practical managerial insights, such as prioritizing models for budget-constrained campaigns and segment-targeting strategies. Overall, e-Profits bridges predictive modeling and profit-driven decision-making, offering a transparent, plug-in evaluation tool for churn management and beyond.

Abstract

Retention campaigns in customer relationship management often rely on churn prediction models evaluated using traditional metrics such as AUC and F1-score. However, these metrics fail to reflect financial outcomes and may mislead strategic decisions. We introduce e-Profits, a novel business-aligned evaluation metric that quantifies model performance based on customer lifetime value, retention probability, and intervention costs. Unlike existing profit-based metrics such as Expected Maximum Profit, which assume fixed population-level parameters, e-Profits uses Kaplan-Meier survival analysis to estimate tenure-conditioned (customer-level) one-period retention probabilities and supports granular, per-customer profit evaluation. We benchmark six classifiers across two telecom datasets (IBM Telco and Maven Telecom) and demonstrate that e-Profits reshapes model rankings compared to traditional metrics, revealing financial advantages in models previously overlooked by AUC or F1-score. The metric also enables segment-level insight into which models maximise return on investment for high-value customers. e-Profits provides a transparent, customer-level evaluation framework that bridges predictive modelling and profit-driven decision-making in operational churn management. All source code is available at: https://github.com/Awaismanzoor/eprofits.

Paper Structure

This paper contains 32 sections, 29 equations, 5 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology and Experimental Setup
  4. Overview and Motivation
  5. e-Profits formulation
  6. Customer-Level Profit Function
  7. Customer Lifetime Value (CLV)
  8. Intervention Cost Model
  9. Retention Offer Cost
  10. Customer Contact Cost
  11. Retention Probability Estimation via Survival Analysis
  12. Comparison with EMP and Implementation Guidance
  13. Computational overhead of Kaplan–Meier
  14. Datasets
  15. Machine Learning Models
  16. ...and 17 more sections

Figures (5)

  • Figure 1: Conceptual workflow of the e-Profits evaluation metric. This flow illustrates how model predictions interact with customer-level retention, CLV, and cost information to compute aggregate business impact. (To calculate CLV, we use ARR and TRR as one-period retention probabilities derived from Kaplan-Meier.)
  • Figure 2: Overview of IBM dataset analysis: retention estimation, churn prioritisation, and classification performance.
  • Figure 3: Overview of Maven dataset analysis: retention estimation, churn prioritisation, and classification performance.
  • Figure 4: Radar graph for the IBM dataset optimised for e-Profits on full population under TRR (axes show normalised scales for AUC, F1, Accuracy, EMP, and e-Profits)
  • Figure 5: Radar graph for Maven dataset optimised for e-Profits on full population under TRR (axes show normalised scales for AUC, F1, Accuracy, EMP, and e-Profits).