ReorderBench: A Benchmark for Matrix Reordering

TL;DR

ReorderBench addresses the lack of diverse, scalable benchmarks for matrix reordering by generating 8.5 million matrices (2{,}835{,}000 binary and 5{,}670{,}000 continuous) across four visual patterns and scoring them with a unified convolution- and entropy-based metric. The benchmark enables rigorous evaluation of existing reordering algorithms, the development of a general scoring model that applies beyond the dataset, and the training of deep reordering models that generalize to unseen matrices. Three main contributions are a scalable template-variation generation pipeline, a robust pattern-scoring framework, and demonstrations of utility in algorithm evaluation, deep scoring, and deep reordering. This resource supports reproducible research and has practical implications for visualization-driven analysis of complex data, including biological and network datasets.

Abstract

Matrix reordering permutes the rows and columns of a matrix to reveal meaningful visual patterns, such as blocks that represent clusters. A comprehensive collection of matrices, along with a scoring method for measuring the quality of visual patterns in these matrices, contributes to building a benchmark. This benchmark is essential for selecting or designing suitable reordering algorithms for specific tasks. In this paper, we build a matrix reordering benchmark, ReorderBench, with the goal of evaluating and improving matrix reordering techniques. This is achieved by generating a large set of representative and diverse matrices and scoring these matrices with a convolution- and entropy-based method. Our benchmark contains 2,835,000 binary matrices and 5,670,000 continuous matrices, each featuring one of four visual patterns: block, off-diagonal block, star, or band. We demonstrate the usefulness of ReorderBench through three main applications in matrix reordering: 1) evaluating different reordering algorithms, 2) creating a unified scoring model to measure the visual patterns in any matrix, and 3) developing a deep learning model for matrix reordering.

Figures (20)

• Figure 1: The generation pipeline for the ReorderBench benchmark.
• Figure 2: The comparison of the structures after filtering: (a) the Robinson structure highlights underlying visual patterns; (b) the non-Robinson structure shows degenerated underlying visual patterns.
• Figure 3: The comparison of generated matrix templates with and without the position constraint: (a) without the constraint, two off-diagonal blocks can merge into a larger block; (b) with the constraint, such merging is prevented.
• Figure 4: The generation of the Robinson structure for visual patterns.
• Figure 5: The two main intrinsic anti-patterns in matrices.