mlr3summary: Concise and interpretable summaries for machine learning models

TL;DR

The paper addresses the need for concise, interpretable, model-agnostic summaries that cover performance, complexity, and explanations across diverse ML models. It introduces mlr3summary, which builds on mlr3 and uses resampling to obtain unbiased estimates of performance, importances, and effects, with outputs that include PDP/PD plots, ALE, and fairness diagnostics via mlr3fairness. The contribution includes a flexible control system, integration with pipelines and AutoTuner, and empirical demonstration on a credit dataset, highlighting improvements over traditional GLM summaries that rely solely on training data. The tool aims to accelerate model selection and auditing by providing structured, comparable summaries across heterogeneous models and tasks.

Abstract

This work introduces a novel R package for concise, informative summaries of machine learning models. We take inspiration from the summary function for (generalized) linear models in R, but extend it in several directions: First, our summary function is model-agnostic and provides a unified summary output also for non-parametric machine learning models; Second, the summary output is more extensive and customizable -- it comprises information on the dataset, model performance, model complexity, model's estimated feature importances, feature effects, and fairness metrics; Third, models are evaluated based on resampling strategies for unbiased estimates of model performances, feature importances, etc. Overall, the clear, structured output should help to enhance and expedite the model selection process, making it a helpful tool for practitioners and researchers alike.