Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting User Experience on Laptops from Hardware Specifications

Saswat Padhi, Sunil K. Bhasin, Udaya K. Ammu, Alex Bergman, Allan Knies

TL;DR

The paper tackles the mismatch between microbenchmarks and real-life UX by predicting user experience on ChromeOS laptops from hardware specifications. It introduces automated ChromeOS webapp tests and curates a dataset of 100K UX data points across 54 Chromebooks, modeling each of 9 Web Vitals–driven UX metrics with Gradient Boosted Regression Trees. The results show strong predictive power, with a mean $R^2=97.8\%$ and a mean $MAAPE=10.1\%$, indicating the approach can accurately map hardware specs to UX. This work enables hardware-aware UX estimation with practical implications for device design and benchmarking beyond traditional microbenchmarks.

Abstract

Estimating the overall user experience (UX) on a device is a common challenge faced by manufacturers. Today, device makers primarily rely on microbenchmark scores, such as Geekbench, that stress test specific hardware components, such as CPU or RAM, but do not satisfactorily capture consumer workloads. System designers often rely on domain-specific heuristics and extensive testing of prototypes to reach a desired UX goal, and yet there is often a mismatch between the manufacturers' performance claims and the consumers' experience. We present our initial results on predicting real-life experience on laptops from their hardware specifications. We target web applications that run on Chromebooks (ChromeOS laptops) for a simple and fair aggregation of experience across applications and workloads. On 54 laptops, we track 9 UX metrics on common end-user workloads: web browsing, video playback and audio/video calls. We focus on a subset of high-level metrics exposed by the Chrome browser, that are part of the Web Vitals initiative for judging the UX on web applications. With a dataset of 100K UX data points, we train gradient boosted regression trees that predict the metric values from device specifications. Across our 9 metrics, we note a mean $R^2$ score (goodness-of-fit on our dataset) of 97.8% and a mean MAAPE (percentage error in prediction on unseen data) of 10.1%.

Predicting User Experience on Laptops from Hardware Specifications

TL;DR

The paper tackles the mismatch between microbenchmarks and real-life UX by predicting user experience on ChromeOS laptops from hardware specifications. It introduces automated ChromeOS webapp tests and curates a dataset of 100K UX data points across 54 Chromebooks, modeling each of 9 Web Vitals–driven UX metrics with Gradient Boosted Regression Trees. The results show strong predictive power, with a mean and a mean , indicating the approach can accurately map hardware specs to UX. This work enables hardware-aware UX estimation with practical implications for device design and benchmarking beyond traditional microbenchmarks.

Abstract

Estimating the overall user experience (UX) on a device is a common challenge faced by manufacturers. Today, device makers primarily rely on microbenchmark scores, such as Geekbench, that stress test specific hardware components, such as CPU or RAM, but do not satisfactorily capture consumer workloads. System designers often rely on domain-specific heuristics and extensive testing of prototypes to reach a desired UX goal, and yet there is often a mismatch between the manufacturers' performance claims and the consumers' experience. We present our initial results on predicting real-life experience on laptops from their hardware specifications. We target web applications that run on Chromebooks (ChromeOS laptops) for a simple and fair aggregation of experience across applications and workloads. On 54 laptops, we track 9 UX metrics on common end-user workloads: web browsing, video playback and audio/video calls. We focus on a subset of high-level metrics exposed by the Chrome browser, that are part of the Web Vitals initiative for judging the UX on web applications. With a dataset of 100K UX data points, we train gradient boosted regression trees that predict the metric values from device specifications. Across our 9 metrics, we note a mean score (goodness-of-fit on our dataset) of 97.8% and a mean MAAPE (percentage error in prediction on unseen data) of 10.1%.
Paper Structure (22 sections, 1 equation, 1 figure, 1 table)

This paper contains 22 sections, 1 equation, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Data Collection
  3. Devices & Specifications.
  4. Workloads & Tests.
  5. User-Experience Metrics.
  6. Methodology
  7. Data Cleaning.
  8. Feature Engineering.
  9. ML Model.
  10. Evaluation
  11. Training.
  12. Assessment.
  13. Limitations
  14. Appendix
  15. Distribution of Hardware Specifications
  16. ...and 7 more sections

Figures (1)

  • Figure 1: The $R^2$ fits and MAAPE errors of our predictors. Dotted lines indicate the mean values.