DECAF: Learning to be Fair in Multi-agent Resource Allocation

TL;DR

This work introduces DECAF, a fairness-enabled extension to the DECA framework for multi-agent resource allocation, where a central ILP allocator coordinates constrained actions based on learned utilities and a fairness objective. It proposes three learning strategies—Joint Optimization (JO), Split Optimization (SO), and Fair-Only Optimization (FO)—implemented on top of Double Deep Q-Learning to balance collective utility and fairness via a per-step fairness reward R_f and a trade-off parameter β in the objective $ (1-β)U_T + βF_T $. Empirically, DECAF methods Pareto-dominate prior fair-MARL baselines across five diverse resource-allocation environments, with SO offering strong online adaptability and FO enabling robust black-box utility integration. The results demonstrate that flexible, decomposed fairness signals can guide long-horizon, constraint-aware decisions and that SO can generalize well to unseen trade-offs, enabling practical real-time fairness-utility tuning in centralized-constrained MARL settings.

Abstract

A wide variety of resource allocation problems operate under resource constraints that are managed by a central arbitrator, with agents who evaluate and communicate preferences over these resources. We formulate this broad class of problems as Distributed Evaluation, Centralized Allocation (DECA) problems and propose methods to learn fair and efficient policies in centralized resource allocation. Our methods are applied to learning long-term fairness in a novel and general framework for fairness in multi-agent systems. We show three different methods based on Double Deep Q-Learning: (1) A joint weighted optimization of fairness and utility, (2) a split optimization, learning two separate Q-estimators for utility and fairness, and (3) an online policy perturbation to guide existing black-box utility functions toward fair solutions. Our methods outperform existing fair MARL approaches on multiple resource allocation domains, even when evaluated using diverse fairness functions, and allow for flexible online trade-offs between utility and fairness.

Figures (12)

• Figure 1: An outline of the DECA pipeline. Each agent evaluates its available actions in a decentralized manner (DE), and the ILP finds the best joint action $\mathcal{A}$ using these evaluations and resource constraints (CA).
• Figure 2: Illustration of our three DECAF methods to learn fairness. Each subfigure shows how the values propagate for a single agent. The red lines and text denote the actual reward to the learning model, which is used to update weights using TD learning. (a) Joint Optimization learns to predict a single combined value. (b) Split Optimization learns two separate estimators for utility and fairness, and combines their output. (c) Fair-Only assumes a black-box utility model $U^*$, and learns a fairness estimator only, combining their outputs to make decisions.
• Figure 3: Change in system utility and fairness as $\beta$ is increased, with $\beta=0$ at the top left $\beta=1$ at the bottom-right. For all domains, we can see that split and joint optimization perform similarly, while learning only fairness can sometimes be slightly worse. All our methods Pareto-dominate SOTO and FEN. Each point depicts the average performance over five different models trained at that $\beta$ value, and the lines show the Pareto front for each method.
• Figure 4: Evaluation of SO models trained on $\beta_{train}$ and evaluated on $\beta_{test}$ for the Matthew environment. Brighter colors indicate better outcomes.
• Figure 5: Evaluation of FO models trained on $\beta_{train}$ and evaluated on $\beta_{test}$ for the Matthew environment. Brighter colors indicate better outcomes.

Theorems & Definitions (9)

• Theorem 4.1
• Theorem 4.2
• Proposition 1.1
• Theorem 1.1
• proof
• Corollary 1.2
• Theorem 1.3
• proof
• Corollary 1.4