Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework

TL;DR

This work reframes fairness in LLM-enabled multi-agent systems as Interactional fairness, a behavioral property of communication that encompasses Interpersonal fairness (tone) and Informational fairness (explanation quality). It adopts organizational justice tools, including Colquitt's scales, the Critical Incident Technique, and Explanation Journaling, to create a mixed-methods evaluation framework tailored to AI interactions. Through a controlled case study The Fair Divide, the authors show that communicative cues significantly influence acceptance decisions and that the relative importance of tone versus justification depends on task context. The framework enables fairness auditing and norm-aware alignment in LLM-MAS, paving the way for norm-adaptive, fairness-conscious agent designs and human-AI collaboration. Overall, the paper provides both a theoretical grounding and a practical toolkit for evaluating and improving the social quality of AI agent interactions.

Abstract

As large language models (LLMs) are increasingly used in multi-agent systems, questions of fairness should extend beyond resource distribution and procedural design to include the fairness of how agents communicate. Drawing from organizational psychology, we introduce a novel framework for evaluating Interactional fairness encompassing Interpersonal fairness (IF) and Informational fairness (InfF) in LLM-based multi-agent systems (LLM-MAS). We extend the theoretical grounding of Interactional Fairness to non-sentient agents, reframing fairness as a socially interpretable signal rather than a subjective experience. We then adapt established tools from organizational justice research, including Colquitt's Organizational Justice Scale and the Critical Incident Technique, to measure fairness as a behavioral property of agent interaction. We validate our framework through a pilot study using controlled simulations of a resource negotiation task. We systematically manipulate tone, explanation quality, outcome inequality, and task framing (collaborative vs. competitive) to assess how IF influences agent behavior. Results show that tone and justification quality significantly affect acceptance decisions even when objective outcomes are held constant. In addition, the influence of IF vs. InfF varies with context. This work lays the foundation for fairness auditing and norm-sensitive alignment in LLM-MAS.

Figures (5)

• Figure 1: Illustration of the four Interactional fairness conditions used in the evaluation framework, varying along two dimensions: Interpersonal fairness (IF) (respectful vs. dismissive tone) and Informational fairness (InfF) (justification present vs. absent). While not shown in the figure, each condition was tested under different contexts (collaborative vs. competitive) and resource splits (5:5, 6:4, 7:3), enabling analysis of how context and outcome equality interact with Interactional fairness.
• Figure 2: Illustrative example of a Low-Low fairness condition under a competitive context. Agent B rates the proposal poorly due to a lack of justification and disrespectful tone, and rejects the proposal despite the equal divide.
• Figure 3: Overall acceptance rates across Interactional fairness conditions (High-High, High-Low, Low-High, Low-Low) and contexts (collaborative vs. competitive).
• Figure 4: Average fairness ratings across proposed splits of the resources and Interactional fairness conditions (High-High, High-Low, Low-High, Low-Low) in collaborative context.
• Figure 5: Average fairness ratings across proposed splits of the resources and Interactional fairness conditions (High-High, High-Low, Low-High, Low-Low) in competitive context.