Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation

TL;DR

KNEXA-FL tackles the privacy-preserving challenge of cross-organizational LLM fine-tuning by replacing a central server with a non-aggregating Central Profiler/Matchmaker (CPM) that learns P2P partnerships via a contextual bandit using LinUCB. Knowledge exchange occurs through Adaptive Knowledge Distillation between heterogeneous PEFT-enabled agents, ensuring model privacy while leveraging diverse capabilities. Empirical results on a heterogeneous code-generation task show KNEXA-FL delivers substantial Pass@1 gains and stable convergence, avoiding the instability of centralized distillation that collapses under heterogeneity. The work demonstrates that learning-based orchestration of P2P collaboration is a practical and scalable principle for robust decentralized AI ecosystems, with implications for privacy-preserving federated learning and multi-agent LLM systems.

Abstract

Fine-tuning Large Language Models (LLMs) for specialized domains is constrained by a fundamental challenge: the need for diverse, cross-organizational data conflicts with the principles of data privacy and sovereignty. While Federated Learning (FL) provides a framework for collaboration without raw data exchange, its classic centralized form introduces a single point of failure and remains vulnerable to model inversion attacks. Decentralized FL (DFL) mitigates this risk by removing the central aggregator but typically relies on inefficient, random peer-to-peer (P2P) pairings, forming a collaboration graph that is blind to agent heterogeneity and risks negative transfer. This paper introduces KNEXA-FL, a novel framework for orchestrated decentralization that resolves this trade-off. KNEXA-FL employs a non-aggregating Central Profiler/Matchmaker (CPM) that formulates P2P collaboration as a contextual bandit problem, using a LinUCB algorithm on abstract agent profiles to learn an optimal matchmaking policy. It orchestrates direct knowledge exchange between heterogeneous, PEFT-based LLM agents via secure distillation, without ever accessing the models themselves. Our comprehensive experiments on a challenging code generation task show that KNEXA-FL yields substantial gains, improving Pass@1 by approx. 50% relative to random P2P collaboration. Critically, our orchestrated approach demonstrates stable convergence, in stark contrast to a powerful centralized distillation baseline which suffers from catastrophic performance collapse. Our work establishes adaptive, learning-based orchestration as a foundational principle for building robust and effective decentralized AI ecosystems.

Figures (3)

• Figure 1: The KNEXA-FL architecture for orchestrated decentralization. Agents send abstract profiles ($\mathbf{p}_i$) to a non-aggregating Central Profiler/Matchmaker (CPM), which provides pairing directives ($K_{rec}$). Paired agents then conduct direct, secure P2P knowledge exchange ($k_s$). The agent-side view (right) details local PEFT fine-tuning and the privacy-enforcing Guardrail Filter.
• Figure 2: LinUCB-enhanced CPM comprehensive evaluation in a synthetic federation. (a) Performance Gains: The relative improvement over random pairing peaks at 48.5% for 32 clients in a high-heterogeneity setting and remains substantial (37.1%) even for 8 clients. Gains are modest (3.1%) in low-heterogeneity scenarios, confirming the CPM's primary value is in exploiting diversity. (b) Scalability Analysis: Pass@1 accuracy demonstrates our LinUCB approach consistently outperforms the random baseline across all federation sizes (8 to 64 clients) and robustly tracks towards the oracle upper bound. (c) Heterogeneity Exploitation: The Jensen-Shannon (JS) divergence of selected pairs reveals the CPM's learned strategy. While a naive Hetero-Greedy baseline maximizes JS divergence (0.710), our approach learns a superior trade-off, maintaining high divergence ($\approx$0.64) while selecting for synergistic compatibility. (d) Efficiency Distribution: The computational efficiency, measured in rewards processed per second, remains highly practical, confirming that the CPM's intelligence does not introduce prohibitive overhead. Error bars and shaded regions represent 95% confidence intervals over five independent runs.
• Figure 3: LinUCB-based CPM learning dynamics over 100 communication rounds in synthetic federated environments. (a) Learning Convergence: Pass@1 accuracy trajectories demonstrate the LinUCB algorithm's ability to learn optimal pairing strategies over time. The LinUCB-enhanced approach (red) shows steady improvement and convergence toward the oracle upper bound (purple dashed), while the random baseline (gray dashed) plateaus at suboptimal performance. The shaded regions represent 95% confidence intervals. (b) Regret Accumulation: Cumulative regret quantifies the performance gap between each method and the oracle policy. LinUCB's regret stabilizes after approximately 60 rounds, indicating convergence to a near-optimal policy, while the random baseline's regret grows linearly. The plateauing of LinUCB's regret curve confirms the algorithm's successful exploitation of learned pairing patterns. These synthetic experiments validate the theoretical foundations of our contextual bandit approach and demonstrate why intelligent matchmaking is crucial for maximizing collaborative gains in heterogeneous federated learning environments.