MANILA: A Low-Code Application to Benchmark Machine Learning Models and Fairness-Enhancing Methods

TL;DR

MANILA tackles the need for accessible fairness-aware benchmarking by introducing a web-based low-code platform powered by an Extended Feature Model (ExtFM) that models a fairness benchmarking workflow as a Software Product Line. It automatically generates executable Python experiments, supports both local and online execution, and reports metric-driven trade-offs to identify the best fairness-effectiveness combination. The authors validate MANILA by reproducing DEMV-based experiments, demonstrating expressiveness and correctness through statistical equivalence to original results and availability of replication artifacts. This work lowers the barrier for practitioners and researchers to evaluate and compare fairness-enhancing strategies without extensive coding, potentially accelerating fairness-aware ML deployment.

Abstract

This paper presents MANILA, a web-based low-code application to benchmark machine learning models and fairness-enhancing methods and select the one achieving the best fairness and effectiveness trade-off. It is grounded on an Extended Feature Model that models a general fairness benchmarking workflow as a Software Product Line. The constraints defined among the features guide users in creating experiments that do not lead to execution errors. We describe the architecture and implementation of MANILA and evaluate it in terms of expressiveness and correctness.

Figures (5)

• Figure 1: MANILA high-level overview
• Figure 2: Fairness metric selection
• Figure 3: Example of web form cross-tree constraints
• Figure 4: Online experiment execution
• Figure 5: Aggregated H-Means of the original experiments and MANILA's ones