Assessing Problem-Solving in HR Contexts: A Comparison Between Game-Based and Self-Report Measures

TL;DR

The paper addresses whether self-perceived problem-solving (PSI-B) converges with observed problem-solving behavior from a game-based assessment (Behaveme-PS) in a personnel-selection context. A method-comparison design was employed with 78 participants who completed PSI-B and a 5-minute Behaveme-PS, with PSI-B scores quartiled into four levels and Behaveme-PS producing four behavioral levels. Results showed no significant convergence between modalities, with $\rho = -0.09$ and $p = 0.452$. The study concludes that self-perceived and behaviorally enacted problem-solving capture distinct facets of the construct and should be integrated in multi-method assessment frameworks to improve decision-making in HR. These findings support incorporating GBAs alongside traditional psychometric instruments to mitigate biases and enhance ecological validity in personnel selection.

Abstract

Game-based assessments (GBAs) are increasingly adopted in recruitment contexts as tools to assess transversal skills through observable behavior. However, empirical evidence directly comparing game-based behavioral indicators with traditional self-report measures remains limited. This study adopts a method-comparison approach to explore the convergence between self-perceived and behaviorally enacted problem-solving competence, comparing a game-based assessment with the Problem Solving Inventory (PSI-B). Seventy-eight participants completed both the PSI-B and a five-minute game-based problem-solving task, which classified performance into four behavioral proficiency levels. Results revealed no significant convergence between self-reported and behavior-based problem-solving scores, indicating a lack of convergence between the two measurement modalities. Rather than indicating a lack of validity of the game-based assessment, these findings support the view that self-report and behavioral measures provide complementary information about problem-solving competence. The study highlights the risks of relying on a single assessment modality in personnel selection and underscores the value of integrating game-based tools within multi-method assessment frameworks.

Figures (2)

• Figure 1: Screenshots of the game-based problem-solving assessment used in the study.
• Figure 2: Distribution of PSI-B scores across the four behavioral groups.