Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Unidimensionality: General Factors and Residual Heterogeneity in Performance Evaluation

Krishna Sharma, Pritam Basnet

TL;DR

Problem addressed: do expert performance evaluations collapse multidimensional information into a single score, or do stable residual dimensions carry meaningful information? Approach: analyze 9,669 professional soccer players rated on 28 attributes using PCA, parallel analysis, bootstrap stability, and holdout validation; compare a PC1 based model to a Ridge model using all attributes. Key findings: PC1 explains $40.6\%$ of variance, four components survive noise discrimination, and residual dimensions are stable and interpretable; out-of-sample prediction with all attributes yields $R^2 = 0.814$, far higher than a PC1-only approach. Significance: results support an interior design regime where moderate compression preserves essential information while identifying stable secondary dimensions, guiding practical measurement reduction and empirical evaluation design.

Abstract

How do evaluation systems compress multidimensional performance information into summary ratings? Using expert assessments of 9,669 professional soccer players on 28 attributes, we characterize the dimensional structure of evaluation outputs. The first principal component explains 40.6% of attribute variance, indicating a strong general factor, but formal noise discrimination procedures retain four components and bootstrap resampling confirms that this structure is highly stable. Internal consistency is high without evidence of redundancy. In out of sample prediction of expert overall ratings, a comprehensive model using the full attribute set substantially outperforms a single-factor summary (cross-validated R squared = 0.814). Overall, performance evaluations exhibit moderate information compression; they combine shared variance with stable residual dimensions that are economically meaningful for evaluation outcomes, with direct implications for the design of measurement systems.

Beyond Unidimensionality: General Factors and Residual Heterogeneity in Performance Evaluation

TL;DR

Problem addressed: do expert performance evaluations collapse multidimensional information into a single score, or do stable residual dimensions carry meaningful information? Approach: analyze 9,669 professional soccer players rated on 28 attributes using PCA, parallel analysis, bootstrap stability, and holdout validation; compare a PC1 based model to a Ridge model using all attributes. Key findings: PC1 explains of variance, four components survive noise discrimination, and residual dimensions are stable and interpretable; out-of-sample prediction with all attributes yields , far higher than a PC1-only approach. Significance: results support an interior design regime where moderate compression preserves essential information while identifying stable secondary dimensions, guiding practical measurement reduction and empirical evaluation design.

Abstract

How do evaluation systems compress multidimensional performance information into summary ratings? Using expert assessments of 9,669 professional soccer players on 28 attributes, we characterize the dimensional structure of evaluation outputs. The first principal component explains 40.6% of attribute variance, indicating a strong general factor, but formal noise discrimination procedures retain four components and bootstrap resampling confirms that this structure is highly stable. Internal consistency is high without evidence of redundancy. In out of sample prediction of expert overall ratings, a comprehensive model using the full attribute set substantially outperforms a single-factor summary (cross-validated R squared = 0.814). Overall, performance evaluations exhibit moderate information compression; they combine shared variance with stable residual dimensions that are economically meaningful for evaluation outcomes, with direct implications for the design of measurement systems.
Paper Structure (15 sections, 6 equations, 6 figures, 7 tables)

This paper contains 15 sections, 6 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Data
  4. Methodology
  5. Internal Consistency
  6. Principal Component Analysis
  7. Parallel Analysis
  8. Bootstrap Stability
  9. Holdout Validation
  10. Exploratory Residual Structure
  11. Results
  12. Discussion
  13. Conclusion
  14. Exploratory Cluster Analysis
  15. Flexible Prediction Benchmark

Figures (6)

  • Figure 1: Scree plot with parallel analysis. Observed eigenvalues (blue) compared to 95th percentile of random noise (red dashed). Four components exceed thresholds; PC5 does not.
  • Figure 2: Bootstrap distribution of PC1 variance (1,000 iterations). The distribution is tightly concentrated around mean 40.6%, with 95% CI [40.0%, 41.1%].
  • Figure 3: Correlation heatmap. Predominantly positive correlations indicate general factor signature, but substantial heterogeneity exists. Defensive attributes (bottom right) show distinct pattern from offensive skills.
  • Figure 4: Holdout prediction scatter (Ridge regression). Predicted overall ratings closely track observed values, indicating strong out-of-sample performance.
  • Figure 5: Silhouette scores by number of clusters. Modest scores indicate continuous structure rather than sharply separated types.
  • ...and 1 more figures