Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale

TL;DR

The paper addresses the limits of monolithic LLM scaling by exploring collective intelligence through orchestrating open-source LLMs. It identifies bottlenecks in routing (text-similarity focus), aggregation (static selection), and missing routing–aggregation synergy, and introduces JiSi with three innovations: Query-Response Mixed Routing, Support-Set-based Aggregator Selection, and Adaptive Routing-Aggregation Switch, all built on a pre-built Embedding Bank. In experiments over nine benchmarks with ten open-source LLMs, JiSi surpasses leading closed-source models like Gemini-3-Pro and achieves substantial cost savings, demonstrating the practical viability of training-free multi-agent collaboration for AI with potential implications for AGI. The results suggest a shift from parameter scaling to collaborative architectures as a scalable, cost-efficient pathway to stronger AI systems.

Abstract

Large Language Models (LLMs) have rapidly advanced, with Gemini-3-Pro setting a new performance milestone. In this work, we explore collective intelligence as an alternative to monolithic scaling, and demonstrate that open-source LLMs' collaboration can surpass Gemini-3-Pro. We first revisit LLM routing and aggregation at scale and identify three key bottlenecks: (1) current train-free routers are limited by a query-based paradigm focusing solely on textual similarity; (2) recent aggregation methods remain largely static, failing to select appropriate aggregators for different tasks;(3) the complementarity of routing and aggregation remains underutilized. To address these problems, we introduce JiSi, a novel framework designed to release the full potential of LLMs' collaboration through three innovations: (1) Query-Response Mixed Routing capturing both semantic information and problem difficulty; (2) Support-Set-based Aggregator Selection jointly evaluating the aggregation and domain capacity of aggregators; (3) Adaptive Routing-Aggregation Switch dynamically leveraging the advantages of routing and aggregation. Comprehensive experiments on nine benchmarks demonstrate that JiSi can surpass Gemini-3-Pro with only 47% costs by orchestrating ten open-source LLMs, while outperforming mainstream baselines. It suggests that collective intelligence represents a novel path towards Artificial General Intelligence (AGI).

Figures (6)

• Figure 1: The overall leaderboard of different LLMs and the proposed JiSi. By orchestrating open-source LLMs collectively, JiSi surpasses all monolithic LLMs, including leading closed-source LLMs such as Gemini-3-Pro, suggesting that collective intelligence represents a novel path towards AGI.
• Figure 2: Comprehensive performance comparison between the proposed JiSi and other LLMs. (a) and (b) show the fine-grained capability comparison with other open-source and closed-source LLMs on different datasets, respectively. JiSi achieves superior performance across multiple benchmarks.
• Figure 3: JiSi rethinks the existing routing and aggregation methods, and reshapes them from three aspects: 1) Routing: from query-based to query-response-mixed; 2) Aggregation: from fixed aggregator to support-set-based aggregator selection; 3) Combination: from static routing and aggregation to adaptive routing-aggregation switch. For simplicity, the "Agg" means Aggregator or Aggregation.
• Figure 4: Overview of the proposed JiSi. For a new question query, a support set is first extracted from the embedding bank based on question similarity. Consequently, three essential techniques, Support-Set-based Aggregator Selection, Query-Response Mixed Routing, and Adaptive Routing-Aggregation Switch, are combined to select and aggregate LLMs dynamically.
• Figure 5: The scaling curve of the proposed JiSi, where we add open-source LLMs following their chronological release sequence as a simulation of the real-world evolution of the LLM ecosystem.