The Model Counting Competition 2020
Johannes K. Fichte, Markus Hecher, Florim Hamiti
TL;DR
The Model Counting Competition 2020 assesses MC, WMC, and PMC solvers on curated CNF benchmarks, detailing data formats, instance sources, and an evaluation harness. It demonstrates the strength of portfolio solvers and knowledge-compilation approaches, while revealing accuracy, reproducibility, and platform challenges that shape future evaluation designs. The results highlight trade-offs between exact and approximate methods, with multiple top performers leveraging combinations of preprocessing, compilation, and DP-like strategies. The work provides a practical snapshot of the state of model counting, proposes improvements to data formats and verification, and aims to catalyze future editions and wider application of counting techniques in probabilistic reasoning and related domains.
Abstract
Many computational problems in modern society account to probabilistic reasoning, statistics, and combinatorics. A variety of these real-world questions can be solved by representing the question in (Boolean) formulas and associating the number of models of the formula directly with the answer to the question. Since there has been an increasing interest in practical problem solving for model counting over the last years, the Model Counting (MC) Competition was conceived in fall 2019. The competition aims to foster applications, identify new challenging benchmarks, and to promote new solvers and improve established solvers for the model counting problem and versions thereof. We hope that the results can be a good indicator of the current feasibility of model counting and spark many new applications. In this paper, we report on details of the Model Counting Competition 2020, about carrying out the competition, and the results. The competition encompassed three versions of the model counting problem, which we evaluated in separate tracks. The first track featured the model counting problem (MC), which asks for the number of models of a given Boolean formula. On the second track, we challenged developers to submit programs that solve the weighted model counting problem (WMC). The last track was dedicated to projected model counting (PMC). In total, we received a surprising number of 9 solvers in 34 versions from 8 groups.