Adaptive Orchestration of Modular Generative Information Access Systems

TL;DR

This paper argues for adaptive, self-organizing orchestration of modular GenIA systems to dynamically configure per-query pipelines from Tasks, Executors, and Resources. It proposes a graph-based framework where pipelines are constructed on-the-fly using planning and learning methods to balance information quality against computational cost. A concrete instantiation using a Contextual Multi-Armed Bandit (CMAB) demonstrates how query complexity can guide the selection of pipeline graphs, achieving favorable efficiency–effectiveness trade-offs. The authors discuss limitations and outline future directions for scaling, dynamic adaptation, and integration of evolving modules, aiming to realize robust, future-ready information access architectures.

Abstract

Advancements in large language models (LLMs) have driven the emergence of complex new systems to provide access to information, that we will collectively refer to as modular generative information access (GenIA) systems. They integrate a broad and evolving range of specialized components, including LLMs, retrieval models, and a heterogeneous set of sources and tools. While modularity offers flexibility, it also raises critical challenges: How can we systematically characterize the space of possible modules and their interactions? How can we automate and optimize interactions among these heterogeneous components? And, how do we enable this modular system to dynamically adapt to varying user query requirements and evolving module capabilities? In this perspective paper, we argue that the architecture of future modular generative information access systems will not just assemble powerful components, but enable a self-organizing system through real-time adaptive orchestration -- where components' interactions are dynamically configured for each user input, maximizing information relevance while minimizing computational overhead. We give provisional answers to the questions raised above with a roadmap that depicts the key principles and methods for designing such an adaptive modular system. We identify pressing challenges, and propose avenues for addressing them in the years ahead. This perspective urges the IR community to rethink modular system designs for developing adaptive, self-optimizing, and future-ready architectures that evolve alongside their rapidly advancing underlying technologies.

Figures (2)

• Figure 1: Example of a graph that represents a system design for a possible pipeline in our framework. Nodes represent modules and are grouped according to their type: Tasks, Executors and Resources, in addition to nodes for the user's input and the output's presentation (GIODBLP:journals/sigir/Culpepper0S18). Edges between tasks represent their order of execution and dependencies, edges from executors indicate which are used to execute tasks, edges from resources indicate which are used for tasks.
• Figure 2: LinUCB expected rewards for the collaborative action space, dashed line depicts real reward for the optimal action.