Revisiting Sparse Matrix Coloring and Bicoloring

TL;DR

The paper tackles efficient computation of sparse Jacobians and Hessians through bidirectional coloring (bicoloring) by reframing the problem with an augmented symmetric matrix H and applying symmetric coloring techniques. It introduces new acyclic and star bicoloring algorithms, plus a post-processing step to neutralize unnecessary colors, and implements them in the Julia package SparseMatrixColorings.jl to outperform traditional ColPack in several scenarios. The approach preserves decompression efficiency (both direct and substitution-based) and leverages partition/graph views to connect colorings with recoverability, offering practical gains for forward/reverse AD workflows. The work demonstrates competitive performance, provides detailed implementation insights, and highlights future directions including orderings, hypergraph connections, and GPU-parallel decompression. All mathematical constructs are presented with formal notation, enabling rigorous reuse in sparse AD contexts.

Abstract

Sparse matrix coloring and bicoloring are fundamental building blocks of sparse automatic differentiation. Bicoloring is particularly advantageous for rectangular Jacobian matrices with at least one dense row and column. Indeed, in such cases, unidirectional row or column coloring demands a number of colors equal to the number of rows or columns. We introduce a new strategy for bicoloring that encompasses both direct and substitution-based decompression approaches. Our method reformulates the two variants of bicoloring as star and acyclic colorings of an augmented symmetric matrix. We extend the concept of neutral colors, previously exclusive to bicoloring, to symmetric colorings, and we propose a post-processing routine that neutralizes colors to further reduce the overall color count. We also present the Julia package SparseMatrixColorings, which includes these new bicoloring algorithms alongside all standard coloring methods for sparse derivative matrix computation. Compared to ColPack, the Julia package also offers enhanced implementations for star and acyclic coloring, vertex ordering, as well as decompression.

Figures (5)

• Figure 1: Row coloring (left) and column coloring (right) of a rectangle matrix, requiring the same number of colors as the matrix dimensions (respectively 6 and 12 in this case).
• Figure 1: Link between symmetric coloring and bicoloring
• Figure 1: Variants of two-colored structures with example matrices.
• Figure 2: Bicoloring of a rectangle matrix, requiring only 2 colors for the rows (left) and 2 colors for the columns (right). In the central figure, each nonzero coefficient is colored using its row's color and its column's color, when it is not neutral.
• Figure 3: Example colorings and bicolorings with their graph representations.