Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

rSRD: An R package for the Sum of Ranking Differences statistical procedure

Balázs R. Sziklai, Attila Gere, Károly Héberger, Jochen Staudacher

TL;DR

This paper presents rSRD, an R package implementing Sum of Ranking Differences (SRD), a non-parametric method for comparing multiple solutions to a fixed reference using the $L_1$ distance on rank-transformed data. It package-composes data preprocessing, SRD calculation, and two validation frameworks—CRRN (permutation-based) and CVST (cross-validation with statistical tests)—into a scalable, reproducible workflow implemented partly in C++ via Rcpp. It demonstrates usage with real datasets (e.g., MEP profiles and Bundesliga statistics) and provides plotting and reference-generation utilities that enhance interpretability and applicability across domains. The work lowers barriers to SRD adoption by practitioners, supports handling of ties and flexible reference construction, and discusses practical options for time-series and large-scale problems.

Abstract

Sum of Ranking Differences (SRD) is a relatively novel, non-para-metric statistical procedure that has become increasingly popular recently. SRD compares solutions via a reference by applying a rank transformation on the input and calculating the distance from the reference in $L_1$ norm. Although the computation of the test statistics is simple, validating the results is cumbersome -- at least by hand. There are two validation steps involved. Comparison of Ranks with Random Numbers, which is a permutation-test, and cross-validation combined with statistical testing. Both options impose computational difficulties albeit different ones. The rSRD package was devised to simplify the validation process by reducing both validation steps into single function calls. In addition, the package provides various useful tools including data preprocessing and plotting. The package makes SRD accessible to a wide audience as there are currently no other software options with such a comprehensive toolkit. This paper aims to serve as a guide for practitioners by offering a detailed presentation of the features.

rSRD: An R package for the Sum of Ranking Differences statistical procedure

TL;DR

This paper presents rSRD, an R package implementing Sum of Ranking Differences (SRD), a non-parametric method for comparing multiple solutions to a fixed reference using the distance on rank-transformed data. It package-composes data preprocessing, SRD calculation, and two validation frameworks—CRRN (permutation-based) and CVST (cross-validation with statistical tests)—into a scalable, reproducible workflow implemented partly in C++ via Rcpp. It demonstrates usage with real datasets (e.g., MEP profiles and Bundesliga statistics) and provides plotting and reference-generation utilities that enhance interpretability and applicability across domains. The work lowers barriers to SRD adoption by practitioners, supports handling of ties and flexible reference construction, and discusses practical options for time-series and large-scale problems.

Abstract

Sum of Ranking Differences (SRD) is a relatively novel, non-para-metric statistical procedure that has become increasingly popular recently. SRD compares solutions via a reference by applying a rank transformation on the input and calculating the distance from the reference in norm. Although the computation of the test statistics is simple, validating the results is cumbersome -- at least by hand. There are two validation steps involved. Comparison of Ranks with Random Numbers, which is a permutation-test, and cross-validation combined with statistical testing. Both options impose computational difficulties albeit different ones. The rSRD package was devised to simplify the validation process by reducing both validation steps into single function calls. In addition, the package provides various useful tools including data preprocessing and plotting. The package makes SRD accessible to a wide audience as there are currently no other software options with such a comprehensive toolkit. This paper aims to serve as a guide for practitioners by offering a detailed presentation of the features.

Paper Structure

This paper contains 18 sections, 1 theorem, 3 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related literature
  3. Ranking transformation in statistical testing
  4. Reference values in statistical testing
  5. Distance metrics in ranking
  6. SRD valildation
  7. SRD
  8. Getting and installing rSRD
  9. Mathematical description of SRD
  10. Validation options
  11. CRRN
  12. CVST
  13. Demonstrative example
  14. Package features
  15. Preprocessing
  16. ...and 3 more sections

Key Result

Theorem 1

(Diaconis_Graham_1977) If $n$ tends to infinity and $D(p,r)$ follows asymptotically normal distribution.

Figures (5)

  • Figure 1: CRRN test: The colored bars from left to right follow the same order as the legend from top to bottom, and their heights are equal to their normalized SRD values, hence expressing the distance from zero. The further they fall from the origin, the less they resemble to the reference. The vertical lines, XX1 and XX19 correspond to the 5% and 95% threshold respectively.
  • Figure 2: Cross-validation: Box whisker plot representing the SRD values calculated on the different folds. The $<$ symbol denotes significant difference between the values, while $\sim$ marks that the null-hypothesis could not be rejected.
  • Figure 3: CVST. Comparing the 8-fold Wilcoxon test with the 10-fold Alpaydin test.
  • Figure 4: CRRN test: SRD values are compared to the cumulative distribution of random rankings generated by method 'r'.
  • Figure 6: CVST. Dietterich test on the MEP profiles. The $<$ symbol denotes significant difference between the values, while $\sim$ marks that the null-hypothesis could not be rejected.

Theorems & Definitions (1)

  • Theorem 1