BEACON: A Benchmark for Efficient and Accurate Counting of Subgraphs
Mohammad Matin Najafi, Xianju Zhu, Chrysanthi Kosyfaki, Laks V. S. Lakshmanan, Reynold Cheng
TL;DR
BEACON delivers a unified benchmark for subgraph counting that enables fair, reproducible comparisons between algorithmic and ML-based methods by providing standardized datasets, verified ground truths, a containerized evaluation environment, and a public leaderboard. It formalizes core notions like local/global, induced/non-induced counts, and error metrics such as Q-Error and MAE, and introduces BEACON-Sampler to curate task-specific datasets. Empirical results reveal complementary strengths: AL methods excel on very large graphs and simple patterns, while ML models better handle larger patterns at the cost of data and sometimes accuracy on dense small graphs, underscoring a clear trade-off landscape. Overall, BEACON advances reproducibility, scalability, and cross-method insights, accelerating progress in subgraph counting research and practical deployments.
Abstract
Subgraph counting the task of determining the number of instances of a query pattern within a large graph lies at the heart of many critical applications, from analyzing financial networks and transportation systems to understanding biological interactions. Despite decades of work yielding efficient algorithmic (AL) solutions and, more recently, machine learning (ML) approaches, a clear comparative understanding is elusive. This gap stems from the absence of a unified evaluation framework, standardized datasets, and accessible ground truths, all of which hinder systematic analysis and fair benchmarking. To overcome these barriers, we introduce BEACON: a comprehensive benchmark designed to rigorously evaluate both AL and ML-based subgraph counting methods. BEACON provides a standardized dataset with verified ground truths, an integrated evaluation environment, and a public leaderboard, enabling reproducible and transparent comparisons across diverse approaches. Our extensive experiments reveal that while AL methods excel in efficiently counting subgraphs on very large graphs, they struggle with complex patterns (e.g., those exceeding six nodes). In contrast, ML methods are capable of handling larger patterns but demand massive graph data inputs and often yield suboptimal accuracy on small, dense graphs. These insights not only highlight the unique strengths and limitations of each approach but also pave the way for future advancements in subgraph counting techniques. Overall, BEACON represents a significant step towards unifying and accelerating research in subgraph counting, encouraging innovative solutions and fostering a deeper understanding of the trade-offs between algorithmic and machine learning paradigms.